Add 'ci' category with two service generators (gitlab, jenkins),
following project conventions (quoted heredocs, PKG_INSTALL, bash_header,
warnings + post_steps). Jenkins WAR URLs pinned in a class dict to
avoid 404s. app.py + generators/__init__.py updated to register.
Dockerfile COPYs start.sh into the image but the project's .dockerignore
had a bare '*.sh' rule that matched it, so 'docker build' failed at:
ERROR [10/11] COPY start.sh /app/start.sh:
CopyIgnoredFile: Attempting to Copy file "start.sh" that is excluded
by .dockerignore (line 50)
The intent of the rule was to skip user-generated deployment scripts
(which shell-gen produces into deliverables/ and /tmp/). A bare '*.sh'
is too broad: it also catches the project's own start.sh.
Fix: narrow to 'deliverables/*.sh' so only user-generated artifacts are
excluded; start.sh and any other project .sh files are now COPY-able.
.dockerignore does not support !-negation, so the right pattern here
is to make the rule path-specific rather than relying on an allowlist.
Verified:
- 'docker build -t shell-gen:0.0.1 .' succeeds through all 11 steps
- container starts, /healthz returns {"generators":50,"ok":true}
- simulated exclude check: start.sh INCLUDED, deliverables/*.sh
EXCLUDED, deliverables/*.txt INCLUDED (unchanged)
Previously the SSL block always generated a self-signed cert at a
hardcoded /etc/nginx/ssl/<server_name>.{crt,key}. That worked for
test/internal use but is unusable for production HTTPS (Let's Encrypt
/ commercial certs): users couldn't point nginx at their own files
without editing the script after generation.
Add three fields (defaults preserve prior behavior):
ssl_cert_source select self_signed|existing
self_signed (default) - existing behavior,
openssl req generates the cert at
/etc/nginx/ssl/<server_name>.{crt,key}.
existing - skip openssl entirely,
use user-supplied ssl_cert_path /
ssl_key_path.
ssl_cert_path text absolute path to fullchain/cert (existing only)
ssl_key_path text absolute path to private key (existing only)
Validation (raised at render time, before script generation):
- ssl_cert_source=existing requires both paths
- both paths must start with '/' (absolute)
Runtime behavior when existing files aren't readable (e.g. certbot
hasn't run yet): script emits a clear WARN with a hint to deploy the
cert first, and continues. nginx -t will fail in that case but the
operator can re-run the script after the cert is in place, or run
'nginx -s reload' directly.
Verified:
- self_signed: full pipeline OK, real openssl produces a valid cert,
real nginx -t passes.
- existing + Let's Encrypt style paths pointing at real certs:
nginx -t passes; no openssl invocation in the script.
- existing + cert files absent: WARN emitted on stderr, script exits 0,
nginx -t skipped (would fail by design — operator runs certbot then
reloads).
- existing with missing path: ValueError at render time.
- existing with relative path: ValueError at render time.
- All 50 generators still pass 'bash -n'.
Two distinct bugs, same root cause: the rendered scripts run with
'set -euo pipefail', but the generator used unquoted heredoc delimiters
(<<CONF_EOF) for config files. Bash expands $vars inside unquoted heredocs;
with 'set -u' any undefined variable aborts the heredoc, and because the
heredoc sits inside 'cat > FILE <<TAG ... TAG', the file gets opened
(truncated to zero bytes) but never written.
1) nginx generator (generators/middleware.py)
- The HTTPS server block ended with '}}\n' instead of '}\n', causing
'nginx -t' to fail with 'unexpected "}"'.
- Both server confs (HTTP + HTTPS) wrote $uri / $host / $scheme /
$request_uri / $proxy_add_x_forwarded_for / $remote_addr into
unquoted heredocs. With set -u the heredoc for harbor.yunwei.blog.conf
aborted on $scheme (and similar), leaving the file empty.
Fix: change both heredoc delimiters to <<'CONF_EOF' (quoted), and
remove the trailing extra '}'. Also strip the unnecessary Python
f-string '{{' / '}}' escapes that produced '}' in the output.
2) mongodb repo (generators/runtimes.py)
- The yum repo file used <<REPO_EOF (unquoted) with a body containing
'$releasever'. $releasever is yum's own variable, not bash's;
bash expanded it to '' under set -u and left the URL broken.
Wireguard/openvpn heredocs DO use $(...) command substitution
intentionally (to inline keys), so those stay unquoted.
Fix: change delimiter to <<'REPO_EOF'.
Verified:
- Rendered nginx.sh for harbor.yunwei.blog (proxy + SSL on port 78/443)
produces conf files with correct nginx syntax. Real 'nginx -t' on a
minimal test config passes (configuration syntax is ok / test is
successful).
- Audit of all 50 generators with default params shows zero $VAR
references inside unquoted heredocs (only intentional $(...) cmds).
- All 50 generators still pass 'bash -n'.
Two changes:
1) Docker generator: introduce 'repo_region' select field with three modes:
auto — probe download.docker.com (5s timeout); fall back to Tuna
mirror if unreachable. Default.
official — force download.docker.com URLs (GPG + apt/yum repo).
cn — force mirrors.tuna.tsinghua.edu.cn/docker-ce URLs.
The apt/yum repo body now uses ${DOCKER_REPO_APT}/${DOCKER_REPO_YUM}
variables instead of hardcoded URLs. yum/dnf uses 'command -v
yum-config-manager' to pick the right tool (dnf-only distros like
Rocky/Alma now skip the failed yum-config-manager probe).
Tuna URLs verified reachable: curl -fsSI returned 200 for both
linux/ubuntu/gpg and linux/centos/gpg. registry_mirror default also
switched to docker.mirrors.ustc.edu.cn (was tencent cloud).
2) shell-gen.service: point WorkingDirectory and ExecStart at
/fs/1000/ftp/Project/shell-gen (the canonical git checkout) instead
of /root/shell-gen. The /root/ copy was a stale duplicate that
caused previous fixes to silently not reach the running web UI.
/root/shell-gen is preserved untouched (user has local artifacts
there: vnc_*.sh, k8s/, deliverables/, etc).
Verified:
- bash -n OK on all 3 region variants
- region=auto with reachable overseas: picks official
- region=auto with overseas-blocked (curl stubbed to fail):
DOCKER_REPO_APT/YUM correctly fall back to tuna.tsinghua,
warning emitted on stderr
- region=official: forces official URLs, no probe
- region=cn: forces Tuna URLs, no probe
- 7 other sampled generators unchanged (memcached, wireguard,
rabbitmq, chrony, mongodb, nginx, redis all bash -n OK,
$PKG_INSTALL count 0)
In bash with 'set -u', an undefined variable triggers an unbound-variable
error. Functions don't satisfy '$FUNCNAME' expansion under set -u, so
the previous fix that kept '$PKG_INSTALL pkg...' as the call syntax
broke every call site:
docker.sh: line 53: PKG_INSTALL: unbound variable
Fix: drop the leading '$' in all 47 call sites across 6 generator
files. 'PKG_INSTALL pkg...' is a normal command/function lookup and
behaves identically under set -u or not.
Verified: web UI served docker.sh now has 'PKG_INSTALL' (no $);
'bash -n' passes; smoke run with stubbed PATH hits real line-53 call
site, mock apt-get receives correct args, exit 0, no unbound variable.
Sampled 6 other generators also syntax-OK with $PKG_INSTALL count 0.
The previous definition assigned a multi-word string to PKG_INSTALL:
PKG_INSTALL="DEBIAN_FRONTEND=noninteractive apt-get install -y"
Downstream generators expand it as '$PKG_INSTALL pkg…', which bash
parses as 'DEBIAN_FRONTEND=noninteractive apt-get install -y pkg…'
— the leading assignment is treated as a command name, producing
'DEBIAN_FRONTEND=noninteractive: command not found' (e.g. docker.sh
line 40). The DEBIAN_FRONTEND variable was also never set, so apt
could fall into interactive mode for tzdata / debconf prompts.
Fix: turn PKG_INSTALL into a shell function and export
DEBIAN_FRONTEND separately. All ~50 call sites keep the existing
$PKG_INSTALL pkg… syntax unchanged.
Verified: bash -n OK; mock Ubuntu 22.04 run exits 0 with no
'command not found'; mock apt-get receives the correct args.
All 44 generators still render successfully.
Ubuntu 22.04 defaults to WaylandEnable=true in /etc/gdm3/custom.conf.
Wayland + x11vnc has known issues (keyboard remap, clipboard, gnome-shell
crashes on X events). Add two options: disable_wayland (write
WaylandEnable=false to gdm3/custom.conf on GNOME installs, guarded when no
gdm3 exists) and restart_gdm (immediately restart gdm vs apply on reboot).
Go official tarball always extracts to <parent>/go. The old code always ran
'mv <parent>/go <install_dir>', which errors with 'cannot move /opt/go to a
subdirectory of itself' when install_dir equals the extraction target (the
default /opt/go). Only emit the mv when install_dir differs from the tar
extraction path.
Both Node.js and Java generators were emitting URLs that 404'd when
selected against the official dist servers.
Node.js: emitted v<major>.x placeholders (e.g. v24.x). nodejs.org's
dist server requires exact version paths (e.g. v24.19.0); the v<x>.x
form is not a redirect — it's a real 404. All 5 supported majors
(18, 20, 22, 24, 26) were broken.
Java: URL was hardcoded to download.java.net's openjdk-21.0.2 GA
build. That path only works for 21; 8, 11, 17 all 404. download.java.net
embeds a build hash in the URL that I don't have a way to look up
for arbitrary patch versions.
Fix:
Node.js: add a LATEST_KNOWN dict mapping each major to its real
current exact version (refreshed 2026-08):
18 -> 18.20.8, 20 -> 20.20.2, 22 -> 22.23.2,
24 -> 24.19.0, 26 -> 26.6.0
Also add a 'download_url' text field with the precise default URL,
so users can override with any mirror (Tuna, npmmirror, etc.) or
a specific patch version without waiting for code to update.
Java: switch to Tuna's Adoptium mirror, whose path format is simple
and predictable:
https://mirrors.tuna.tsinghua.edu.cn/Adoptium/<major>/jdk/x64/linux/OpenJDK<major>U-jdk_x64_linux_hotspot_<exact>.tar.gz
This works for 8/11/17/21 with the LATEST_TUNA mapping. Also add a
'download_url' text field for overrides.
Both also derive the tarball's inner directory name from the URL (via
regex) so the --strip-components=1 trick keeps working even when the
user picks a custom URL.
Verified all 9 generated URLs return 200 (HEAD):
Node v18/20/22/24/26: all 200
Java v8/11/17/21: all 200 (Tuna mirror)
All 50 generators still pass bash -n.
Major changes to the VNC generator (generators/middleware.py):
1. Replace Xtigervnc + vncserver wrapper with xvfb + x11vnc.
- Xtigervnc defaults to daemon mode (no -fg flag exists), so systemd
Type=simple couldn't track it; x11vnc attaches to Xvfb in
foreground, works cleanly with Type=simple.
- Use display :99 via xvfb-run -a so we never collide with the
physical GDM/Xorg on :0 (avoids 'server already running').
2. GNOME xstartup: switch to Ubuntu's canonical session.
- XDG_CURRENT_DESKTOP=ubuntu:GNOME (was 'GNOME'; the bare form
doesn't match Ubuntu's /usr/share/gnome-session/sessions/ entry
and apps like gnome-terminal refuse to launch).
- XDG_SESSION_TYPE=x11 (x11vnc is X11-only, so be explicit).
- GNOME_SHELL_SESSION_MODE=ubuntu (Ubuntu-patched gnome-shell hint).
3. Install extra GNOME packages needed for a usable VNC desktop.
ubuntu-desktop-minimal alone leaves the user with a blank desktop
and no way to launch apps. Add:
- gnome-shell-extension-desktop-icons-ng (desktop icons)
- gnome-terminal (terminal app)
- nautilus-extension-gnome-terminal (right-click → open in terminal)
- gnome-tweaks (advanced settings UI)
4. Two new UI options:
- 'Disable Unix socket (-nolisten unix)' — default on (avoids GDM
collision on /tmp/.X11-unix/X0).
- 'Disable X11 TCP listener (6001)' — default off (frees 6001 when
GDM is squatting on it).
5. .gitignore: exclude other projects' deliverables/ directory and
a few test output scripts (vnc_*.sh).
Your Name