Debugging 6.5 Million PM2 Restarts: Stabilizing MyZubster Sepolia and I-ECO-01
Today I completed a significant infrastructure debugging session across the MyZubster Sepolia E2E environment and the I-ECO-01 / UrbanLab backend.
What initially looked like a few unrelated configuration problems turned into a useful example of why production debugging needs to happen layer by layer.
The work involved Node.js, PM2, MongoDB, Google OAuth 2.0, Cloudflare, Socket.IO, and a Linux VPS.
Here is what happened, what was broken, and how it was fixed.
1. Google OAuth Was Reaching the Backend — But Authentication Still Failed
The first problem appeared in the Google social-login flow.
The authorization request itself looked correct.
Google was receiving the expected client ID and redirect URI:
https://sepolia.myzubster.com/api/auth/social/google/callback
The callback also reached the backend correctly.
Debug instrumentation confirmed:
hasCode: true
hasState: true
hasError: false
So the browser redirect, public route, OAuth state and callback routing were all working.
The failure happened one step later: during the authorization-code exchange.
The backend logged:
401 invalid_client
and Google reported that the provided client secret was invalid.
That was an important distinction.
The problem was not the redirect URI.
It was not Cloudflare.
It was not Express routing.
It was not the OAuth state.
It was not the frontend.
The OAuth client credential itself was wrong.
After correcting the configuration, the backend reached:
OAUTH-CALLBACK-ENTER
followed by:
OAUTH-CALLBACK-SUCCESS
The server-side Google OAuth flow was working.
2. The .env File Was Correct — The Running Process Was Not
The next issue was more subtle.
The .env file contained:
FRONTEND_URL=https://sepolia.myzubster.com
and:
GATEWAY_PUBLIC_URL=https://sepolia.myzubster.com
Yet after successful OAuth authentication, the application could still redirect toward an old Vercel deployment.
This highlighted an important operational detail:
Changing .env does not automatically mean a long-running process is using the new values.
I inspected the configuration at several levels:
.env
- Node + dotenv
- PM2 process configuration
- actual process environment
- OAuth redirect generation
- backend logs
The active PM2 environment was then aligned with the Sepolia deployment:
FRONTEND_URL=https://sepolia.myzubster.com
PUBLIC_APP_URL=https://sepolia.myzubster.com
GATEWAY_PUBLIC_URL=https://sepolia.myzubster.com
After that, the active process environment was verified directly.
3. Then I Found a Process With More Than 6.5 Million Restarts
While checking the rest of the VPS, another issue appeared.
The UrbanLab / I-ECO-01 PM2 process was shown as online, but its restart counter was enormous:
6,548,384 restarts
That immediately suggested a crash/restart loop.
This is one of the reasons I do not treat:
PM2 status = online
as proof that an application is healthy.
A process can start, crash immediately, restart automatically and still repeatedly appear online when inspected at the wrong moment.
4. Finding the Root Cause
PM2 was executing:
/opt/I-ECO-01/server.js
I inspected the server entrypoint and found these dependencies:
dotenv
express
cors
helmet
mongoose
http
socket.io
winston
Then I compared those requirements against package.json.
The project only declared:
cors
dotenv
express
mongoose
The logs confirmed the immediate failure:
Error: Cannot find module 'helmet'
The server crashed at:
const helmet = require('helmet');
Because PM2 automatically restarted the process, the cycle repeated continuously.
5. Fixing the Dependency Graph
Instead of installing only the first missing package and waiting for the next crash, I inspected all external imports in the server entrypoint.
Three npm dependencies were missing:
helmet
socket.io
winston
http did not require installation because it is part of Node.js itself.
The missing dependencies were installed and persisted to package.json.
The resulting application dependencies now include:
- Express
- CORS
- dotenv
- Helmet
- Mongoose
- Socket.IO
- Winston
Each new module was then tested independently with require() before restarting the service.
All modules loaded successfully.
6. Application Startup Was Not Enough — I Tested Stability
After restarting UrbanLab, the application successfully initialized its Escrow controller.
The server reported:
Server started on port 5002
MongoDB connected successfully.
The health endpoint:
http://127.0.0.1:5002/health
returned:
status: ok
service: I-ECO-01
mongodb: connected
But I still did not consider the problem solved.
The original issue was a restart loop, so the correct test was to verify process stability over time.
Before the stability test:
PID = 1026452
restarts = 6548384
After 15 seconds:
PID = 1026452
restarts = 6548384
Same PID.
Same restart counter.
The loop had stopped.
7. Escrow Infrastructure Is Running Again
Once the dependency problem was resolved, the I-ECO-01 Escrow controller initialized with:
Threshold: 2/3
Timeout: 24 hours
Fee: 0.5%
The service also initialized its HTTP API and WebSocket infrastructure.
MongoDB remained connected and the health endpoint continued responding successfully.
So the service moved from an uncontrolled crash loop to an operational backend.
8. Old Logs Can Be Misleading
One interesting detail appeared after the repair.
Running:
pm2 logs urbanlab
still displayed:
Cannot find module 'helmet'
At first glance, that could suggest that the problem still existed.
But those entries were historical.
The current process had already successfully loaded Helmet, initialized the application, connected to MongoDB and remained alive with the same PID.
This is why timestamps, process IDs, restart counters and health checks matter when reading operational logs.
A log file can contain a real error that is no longer a current error.
9. What I Verified
By the end of the session:
Google OAuth provider: operational
Google OAuth callback: operational
OAuth authorization-code exchange: successful
OAuth state handling: verified
Sepolia public URL configuration: corrected
PM2 runtime environment: verified
I-ECO-01 Node.js dependencies: repaired
UrbanLab restart loop: stopped
MongoDB: connected
Health endpoint: healthy
Escrow controller: initialized
2-of-3 multisig configuration: loaded
Socket.IO: initialized
PM2/systemd environment: verified
10. The Main Lessons
This debugging session reinforced several useful principles.
online does not mean healthy
A process manager tells you about process state.
An application health endpoint tells you about application state.
You often need both.
.env does not equal runtime environment
Always verify what the running process actually sees.
This is especially important when using PM2, containers, systemd or other long-running process managers.
Debug OAuth one stage at a time
OAuth involves multiple independent stages:
authorization request → provider consent → callback → state verification → authorization-code exchange → profile retrieval → application session → frontend redirect.
Logging each boundary makes failures much easier to isolate.
Fix dependency problems systematically
When one require() fails, inspect the entire entrypoint instead of repeatedly installing dependencies one crash at a time.
Verify the original failure condition
The problem was a restart loop.
Therefore, successful startup alone was not enough.
The actual success condition was:
same PID + unchanged restart counter + healthy endpoint over time.
Final Result
The MyZubster Sepolia environment is now significantly more stable.
Google OAuth can successfully complete its backend authentication flow.
The Sepolia runtime URLs are correctly propagated to the active backend process.
I-ECO-01 is connected to MongoDB.
The Escrow service is initialized.
And, most importantly, a Node.js service that had accumulated more than 6.5 million PM2 restarts is now staying alive.
Sometimes the most useful development work is not adding another feature.
It is understanding exactly why the existing system is failing — and proving that the fix actually removed the failure.
Stack: Node.js · Express · MongoDB · PM2 · Google OAuth 2.0 · Socket.IO · Cloudflare · Linux · Sepolia
NodeJS #DevOps #WebDevelopment #BackendDevelopment #Debugging #PM2 #MongoDB #OAuth2 #GoogleOAuth #SocketIO #Web3 #Sepolia #Cloudflare #Linux #MyZubster #CoderLegion