When My Tests Worked Locally but CI Couldn’t Find My Code: Fixing PYTHONPATH in CodeBots Arena
This is a submission for DEV’s Summer Bug Smash: Smash Stories powered by Sentry.
The Bug
While developing CodeBots Arena, an AI vs AI code battle platform, I started building automated tests for the battle engine.
The tests worked in my local development environment.
Then I moved the same test workflow into CI.
That is where things became interesting.
The CI environment could not resolve the project modules correctly. The code was there, the tests were there, but Python was not looking in the right place when the tests were executed.
The problem came down to the Python module search path.
Local Environment vs. CI
Locally, my development environment already had the project structure and paths configured in a way that allowed imports to work.
CI was a clean environment.
That difference exposed an assumption in the project:
“The test runner knew where to find the server-side Python package.”
It did not.
This is one of those bugs that can be confusing because the code itself looks correct.
The failure isn’t necessarily caused by the function being tested. It can happen before the test even gets to that point.
Tracking Down the Problem
I started by looking at the test imports and the project structure.
The tests needed to import components such as the battle engine, but the CI environment did not automatically include the required project directory in Python’s module search path.
Instead of changing the tests just to make them pass, I wanted the CI environment to correctly understand the project structure.
That led me to PYTHONPATH.
️ The Fix
I updated the GitHub Actions test step so that the project’s server directory was included in PYTHONPATH before running pytest.
The CI workflow effectively changed from simply running the tests to explicitly telling Python where the project modules were located.
The important part was:
env:
PYTHONPATH: ${{ github.workspace }}/server
The tests could then be executed with:
python -m pytest tests/ -v
This change is recorded in the repository history in the commit:
fa401d4 — Add PYTHONPATH to test run step in CI workflow
✅ Verification
After the change, the CI test environment could resolve the required Python modules correctly.
The test workflow was able to move past the import/path problem and execute the automated tests.
This was a good reminder that “works on my machine” doesn’t always mean “works in CI.”
A clean CI environment can expose assumptions that are invisible during local development.
What I Learned
The biggest lesson wasn’t just how to set PYTHONPATH.
It was learning to distinguish between an application bug and an environment/configuration bug.
When a test fails, I now try to answer:
- Did the application logic fail?
- Did the test fail?
- Or did the environment fail before the test could even run?
That simple distinction can save a lot of debugging time.
The Challenge
The challenging part was that the project itself was available in the CI environment.
The problem was not that the files were missing.
Python simply didn’t have the correct location in its module search path.
That made the issue easy to overlook if I focused only on the application code.
Before vs. After
Before:
The tests worked in my local development environment, but the CI environment could not correctly resolve the project’s Python modules.
After:
The CI workflow explicitly configured PYTHONPATH to include the server directory, allowing the test runner to locate the required modules and execute the test suite.
Why This Bug Matters
This was a relatively small configuration change, but it highlighted an important software engineering principle:
Development environments should not hide assumptions from CI.
Automated testing is only useful when the environment running those tests is configured correctly.
For CodeBots Arena, where the battle engine and AI bot components depend on multiple Python modules, making the CI environment predictable is an important part of keeping the project maintainable.
What I’m Proud Of
I’m proud that the fix was small and targeted.
I didn’t rewrite the test suite or change application logic just to satisfy CI.
I identified the actual environment problem and fixed the configuration at the appropriate layer.
Sometimes the best bug fix is the smallest change that addresses the real cause.
Project
CodeBots Arena is an AI vs AI Code Battle Arena built around automated battles between coding agents.
The project includes a battle engine, bot SDK, ELO rating system, isolated execution, API components, and automated tests.
GitHub:
https://github.com/Mhmda1998/CodeBots-Arena
Final Takeaway
A test passing locally can give you confidence, but CI gives you a different perspective.
In my case, CI exposed an assumption that my local environment had been hiding.
The fix was one configuration change, but the lesson was much bigger:
“When CI says it can’t find your code, don’t immediately rewrite the code. Check the environment first.”
devchallenge #bugsmash #python #githubactions #testing #debugging #ai #softwaredevelopment
