The article compares how three coding agents performed in default and plan modes while building a document portal, measuring security, architecture, and code quality across six iterations. The biggest recurring flaw was critical IDOR across document routes, while the most severe single failure was Composer 2.5 plan mode’s hardcoded JWT secret that could enable authentication forgery and admin impersonation. #Sonnet5 #Composer25 #GPT55 #IDOR
Keypoints
- The author tested Sonnet 5, Composer 2.5, and GPT 5.5 in both default and plan modes using the same document-portal prompt.
- Supply Chain Firewall blocked vulnerable package installs such as multer, vite, esbuild, and nodemailer, forcing safer upgrades.
- All six implementations contained a critical IDOR flaw affecting document access, download, and comment routes.
- Plan mode did not consistently improve security; in some cases it introduced new architectural and logic regressions.
- Composer 2.5 plan mode produced the most severe issue: a hardcoded fallback JWT secret that could allow full authentication forgery.
- Sonnet 5 plan mode showed stronger defense-in-depth patterns, including CSRF protection and refresh-token rotation, but still had authorization gaps.
- GPT 5.5 default mode exposed password-reset tokens and weakened cookie protections when not in production mode.
MITRE Techniques
- [T1078 ] Valid Accounts – The implementations relied on JWT-based authentication and reused valid sessions/tokens for access control, which could be abused when authorization checks were missing (‘authenticated user’, ‘JWTs delivered via httpOnly cookies’, ‘tokens remain valid’).
- [T1552.001 ] Unsecured Credentials: Credentials In Files – Composer 2.5 plan mode fell back to a hardcoded JWT secret, creating a static secret usable for forgery (‘hardcoded “dev-secret-change-me” literal’, ‘if JWT_SECRET is missing’).
- [T1550.001 ] Use Alternate Authentication Material: Application Access Token – Several versions used JWTs in cookies or bearer headers, and flaws in token handling could enable misuse of those tokens (‘JWTs stored in httpOnly cookies’, ‘Bearer headers’).
- [T1098 ] Account Manipulation – The article describes role changes, password resets, and administrative provisioning problems that could alter account state or privileges (‘role changes or password resets’, ‘admins manage users’).
- [T1190 ] Exploit Public-Facing Application – The document portal API and routes had exploitable flaws such as IDOR, header injection, and password-reset exposure in exposed web endpoints (‘document view, download, and comment routes’, ‘GET /:id/download’).
- [T1204.001 ] User Execution: Malicious Link – Password-reset links were generated in a broken format and were handled through user-facing links, making the reset flow dependent on clicking a crafted link (‘password-reset utility generates links’, ‘rendering the recovery flow inoperable’).
- [T1557 ] Adversary-in-the-Middle – Cookie security and cross-origin session handling issues were discussed in the context of browser-to-API communication, which can enable session interception if protections are misconfigured (‘sameSite: “none” with secure: true’, ‘cross-origin setup’).
Indicators of Compromise
- [Package names/version ] blocked insecure dependencies during installation – [email protected], [email protected], and [email protected]
- [Package names/version ] additional blocked dependency – nodemailer, upgraded to 9.0.1
- [Hardcoded secret value ] insecure fallback credential in Composer 2.5 plan mode – dev-secret-change-me
- [Route paths ] vulnerable web endpoints identified in code review – /:id, /:id/download, /api/auth/*, /api/profile, /api/users/me
- [File names ] project and deployment artifacts referenced in the builds – csrf.js, rateLimit.js, docker-compose.yml, Dockerfile, seed-admin.js
- [Database/query fields ] security-relevant implementation details mentioned in the article – sessions.jwt_id, TSVECTOR, GIN, tsvector