Most of the machine learning posts here are about training. This one is about the part after training, when the model has to answer over HTTP and you would rather not rent that by the request.
The usual path is to save the trained artifact, put FastAPI in front of model.predict(), containerize it and run it somewhere. That works, and it also hands you a list of chores: pinning the training and serving environments to matching library versions, versioning artifacts, writing your own retraining trigger, and building the admin nobody wants to build so a non engineer can add examples.
What A Self Hosted Prediction Server Changes
The alternative is running a small server that already owns the whole loop. ML Prediction Engine is the one I maintain, MIT licensed, a PHP front end with a Python brain. You create a pipeline, choose a model class from a catalog of 18, add labeled example rows either in the browser admin or through the API, hit train, and the predict endpoint answers from that moment on.
curl -X POST https://yourserver.com/api.php/predict \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_KEY" \
-d '{"pipelineID": 1, "input": "why was i charged twice"}'
Answer shapes follow the model category. Classifiers return the label, regressors return a number, clusterers return a cluster id with DBSCAN using -1 for noise, and anomaly detectors return 1 or 0.
The Two Design Choices That Matter
The dataset stays visible. Every model's training rows live in a jsonL file and a readable table in the admin, so you can audit exactly what the model learned from, prune rows you want gone, and retrain by hand. A model you cannot regenerate becomes a liability the first time a label definition changes.
Live training is a per pipeline switch. With it on, every prediction is appended to the dataset labeled with its own output, and the step rebuilds itself after a number of new rows you set. That fits feedback clustering and anomaly baselines, where the right answer keeps moving. It is the wrong default for a classifier whose labels need to stay stable, which is why it stays opt in.
Nothing leaves your server, there is no per request fee, and the source is on GitHub. Worth a look if you have a prediction job sitting in a backlog because it looked like a quarter of work.