Docker backend health status issue #879
Description
port 8000 and 3000 is already in use so instead i replaced them
8001 => 8082
8000 => 8083
3000 => 8084
oasis@wpnl:~/docker$ git clone https://github.com/PromtEngineer/localGPT.git
Cloning into 'localGPT'...
remote: Enumerating objects: 1328, done.
remote: Counting objects: 100% (104/104), done.
remote: Compressing objects: 100% (63/63), done.
remote: Total 1328 (delta 59), reused 44 (delta 41), pack-reused 1224 (from 4)
Receiving objects: 100% (1328/1328), 4.59 MiB | 4.14 MiB/s, done.
Resolving deltas: 100% (639/639), done.
oasis@wpnl:~/docker$ cd localGPT/
oasis@wpnl:~/docker/localGPT$ grep -rl --exclude=*.db '8001' .
./docker-compose.yml
./setup_rag_system.sh
./backend/server.py
./test_docker_build.sh
./src/lib/api.ts
./Dockerfile.rag-api
./run_system.py
./Documentation/quick_start.md
./Documentation/api_reference.md
./Documentation/installation_guide.md
./Documentation/docker_usage.md
./Documentation/system_overview.md
./Documentation/deployment_guide.md
./README.md
./DOCKER_README.md
./docker.env
./DOCKER_TROUBLESHOOTING.md
./start-docker.sh
./rag_system/api_server.py
./rag_system/api_server_with_progress.py
./docker-compose.local-ollama.yml
oasis@wpnl:~/docker/localGPT$ grep -rl --exclude=*.db '8000' .
./docker-compose.yml
./setup_rag_system.sh
./backend/server.py
./backend/README.md
./backend/test_backend.py
./test_docker_build.sh
./src/lib/api.ts
./src/test-upload.html
./Dockerfile.backend
./run_system.py
./Documentation/quick_start.md
./Documentation/api_reference.md
./Documentation/installation_guide.md
./Documentation/docker_usage.md
./Documentation/system_overview.md
./Documentation/deployment_guide.md
./README.md
./DOCKER_README.md
./docker.env
./DOCKER_TROUBLESHOOTING.md
./start-docker.sh
./rag_system/api_server.py
./rag_system/api_server_with_progress.py
./docker-compose.local-ollama.yml
oasis@wpnl:~/docker/localGPT$ grep -rl --exclude=*.db '3000' .
./docker-compose.yml
./setup_rag_system.sh
./backend/test_backend.py
./test_docker_build.sh
./run_system.py
./Documentation/quick_start.md
./Documentation/installation_guide.md
./Documentation/docker_usage.md
./Documentation/system_overview.md
./Documentation/deployment_guide.md
./README.md
./package-lock.json
./DOCKER_README.md
./DOCKER_TROUBLESHOOTING.md
./start-docker.sh
./Dockerfile.frontend
./docker-compose.local-ollama.yml
oasis@wpnl:~/docker/localGPT$ grep -rl --exclude=*.db '8000' . | xargs sed -i 's/8000/8083/g'
oasis@wpnl:~/docker/localGPT$ grep -rl --exclude=*.db '8001' . | xargs sed -i 's/8001/8082/g'
oasis@wpnl:~/docker/localGPT$ grep -rl --exclude=*.db '3000' . | xargs sed -i 's/3000/8084/g'
oasis@wpnl:~/docker/localGPT$ ollama list
NAME ID SIZE MODIFIED
qwen3:8b 500a1f067a9f 5.2 GB 5 days ago
qwen3:0.6b 7df6b6e09427 522 MB 5 days ago
oasis@wpnl:~/docker/localGPT$ ollama serve
Error: listen tcp 127.0.0.1:11434: bind: address already in use
Edit docker-compose.yml with actual network name to prevent docker from prepending the project (usually the folder) name to the network name
networks:
rag-network:
driver: bridge
name: rag-network
oasis@wpnl:~/docker/localGPT$ docker compose --env-file docker.env up --build -d
[+] Running 7/7
✔ rag-api Built 0.0s
✔ backend Built 0.0s
✔ frontend Built 0.0s
✔ Network rag-network Created 0.1s
✔ Container rag-api Healthy 47.6s
✘ Container rag-backend Error 199.4s
✔ Container rag-frontend Created 0.1s
dependency failed to start: container rag-backend is unhealthy
oasis@wpnl:~/docker/localGPT$ docker compose logs
rag-backend | /app/backend/server.py:4: DeprecationWarning: 'cgi' is deprecated and slated for removal in Python 3.13
rag-backend | import cgi
rag-api | 2025-07-20 07:10:41,087 | INFO | rag_system.indexing.contextualizer | Initialized ContextualEnricher with Ollama model 'qwen3:0.6b' (batch_size=10).
rag-api | 127.0.0.1 - - [20/Jul/2025 07:10:46] "GET /models HTTP/1.1" 200 -
rag-api | 127.0.0.1 - - [20/Jul/2025 07:11:21] "GET /models HTTP/1.1" 200 -
rag-api | 127.0.0.1 - - [20/Jul/2025 07:11:56] "GET /models HTTP/1.1" 200 -
rag-api | 127.0.0.1 - - [20/Jul/2025 07:12:31] "GET /models HTTP/1.1" 200 -
rag-api | 127.0.0.1 - - [20/Jul/2025 07:13:06] "GET /models HTTP/1.1" 200 -
rag-api | 127.0.0.1 - - [20/Jul/2025 07:13:42] "GET /models HTTP/1.1" 200 -
oasis@wpnl:~/docker/localGPT$ docker compose ps -a
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
rag-api localgpt-rag-api "python -m rag_syste…" rag-api 5 minutes ago Up 5 minutes (healthy) 0.0.0.0:8082->8082/tcp, [::]:8082->8082/tcp
rag-backend localgpt-backend "python server.py" backend 5 minutes ago Up 4 minutes (unhealthy) 0.0.0.0:8083->8083/tcp, [::]:8083->8083/tcp
rag-frontend localgpt-frontend "docker-entrypoint.s…" frontend 5 minutes ago Created
oasis@wpnl:~/docker/localGPT$ curl -v http://localhost:8083/health
* Host localhost:8083 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
* Trying [::1]:8083...
* Connected to localhost (::1) port 8083
> GET /health HTTP/1.1
> Host: localhost:8083
> User-Agent: curl/8.5.0
> Accept: */*
>
oasis@wpnl:~/docker/localGPT$ docker exec -it rag-backend /bin/bash
root@6672ee1f3b5c:/app/backend# ps aux
bash: ps: command not found
root@6672ee1f3b5c:/app/backend# apt update && apt install -y net-tools procps curl
Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
...
root@6672ee1f3b5c:/app/backend# netstat -tlnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.11:45205 0.0.0.0:* LISTEN -
tcp 6 0 0.0.0.0:8083 0.0.0.0:* LISTEN 1/python
root@6672ee1f3b5c:/app/backend# exit
exit
- The backend container is listening on port 8083 (
netstatshows0.0.0.0:8083 LISTEN 1/python). - But curl to
/healthinside the container hangs (no response, not even a connection refused). - Running
python3 server.pymanually givesOSError: [Errno 98] Address already in use(because the main process is already running and bound to 8083).
This means:
- The backend process is running and listening, but not responding to HTTP requests (at least not to
/health). - The healthcheck fails, so Docker marks the container as unhealthy.
After several back an forth, i discovered a fixed that works manually by making some changes to backend/server.py
# Test Ollama connection
client = OllamaClient()
if client.is_ollama_running():
models = client.list_models()
print(f"✅ Ollama is running with {len(models)} models")
print(f"📋 Available models: {', '.join(models[:3])}{'...' if len(models) > 3 else ''}")
else:
print("⚠️ Ollama is not running. Please start Ollama:")
print(" Install: https://ollama.ai")
print(" Run: ollama serve")
print(f"\n🌐 Frontend should connect to: http://localhost:{PORT}")
print("💬 Ready to chat!\n")
with ReusableTCPServer(("", PORT), ChatHandler) as httpd:
httpd.serve_forever()
except KeyboardInterrupt:
print("\n🛑 Server stopped")
if __name__ == "__main__":
main()
ignore the Ollama error,
Instead of letting Docker start the backend, I stopped the container and run the server manually in the container and also check for the health status simultaneously as well
This shows that curl to http://localhost:8083/health failed, which may be the reason docker health check status failed as well.
oasis@wpnl:~/docker/localGPT$ docker compose stop backend
[+] Stopping 1/1
✔ Container rag-backend Stopped 10.8s
oasis@wpnl:~/docker/localGPT$ docker run -it --rm --network rag-network -p 8083:8083 localgpt-backend /bin/bash -c "curl -v http://localhost:8083/health && exit"
* Trying 127.0.0.1:8083...
* connect to 127.0.0.1 port 8083 failed: Connection refused
* Trying [::1]:8083...
* connect to ::1 port 8083 failed: Connection refused
* Failed to connect to localhost port 8083 after 0 ms: Couldn't connect to server
* Closing connection 0
curl: (7) Failed to connect to localhost port 8083 after 0 ms: Couldn't connect to server
Then I figured i have to bash in first and start the server and health check curl simultaneously but with 30s delay, as intended in the Dockerfile.backend.
oasis@wpnl:~/docker/localGPT$ docker run -it --rm --network rag-network -p 8083:8083 localgpt-backend /bin/bash
root@952bd13803f1:/app/backend# python3 server.py & sleep 30 && curl -v http://localhost:8083/health & fg
[1] 28
[2] 29
sleep 30 && curl -v http://localhost:8083/health
/app/backend/server.py:4: DeprecationWarning: 'cgi' is deprecated and slated for removal in Python 3.13
import cgi
✅ RAG system modules accessible from backend
✅ Database initialized successfully
✅ Database initialized successfully
✅ Simple PDF processor initialized
✅ Global PDF processor initialized
📄 Initializing simple PDF processing...
✅ Simple PDF processor initialized
✅ Global PDF processor initialized
🧹 Cleaning up empty sessions...
✨ No empty sessions to clean up
🚀 Starting localGPT backend server on port 8083
📍 Chat endpoint: http://localhost:8083/chat
🔍 Health check: http://localhost:8083/health
⚠️ Ollama is not running. Please start Ollama:
Install: https://ollama.ai
Run: ollama serve
🌐 Frontend should connect to: http://localhost:8083
💬 Ready to chat!
Error fetching models: HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url: /api/tags (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eeb2cdbdd90>: Failed to establish a new connection: [Errno 111] Connection refused'))
[Sun, 20 Jul 2025 08:39:02 GMT] "GET /health HTTP/1.1" 200 -
* Trying 127.0.0.1:8083...
* Connected to localhost (127.0.0.1) port 8083 (#0)
> GET /health HTTP/1.1
> Host: localhost:8083
> User-Agent: curl/7.88.1
> Accept: */*
>
Error fetching models: HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url: /api/tags (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eeb2cd69d50>: Failed to establish a new connection: [Errno 111] Connection refused'))
[Sun, 20 Jul 2025 08:39:18 GMT] "GET /health HTTP/1.1" 200 -
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Server: BaseHTTP/0.6 Python/3.11.13
< Date: Sun, 20 Jul 2025 08:39:18 GMT
< Content-Type: application/json
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
< Access-Control-Allow-Headers: Content-Type, Authorization
< Access-Control-Allow-Credentials: true
<
{
"status": "ok",
"ollama_running": false,
"available_models": [],
"database_stats": {
"total_sessions": 0,
"total_messages": 0,
"most_used_model": null
}
* Closing connection 0
}root@952bd13803f1:/app/backend# Error fetching models: HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url: /api/tags (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eeb2cdbd1d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
[Sun, 20 Jul 2025 08:39:32 GMT] "GET /health HTTP/1.1" 200 -
^C
root@952bd13803f1:/app/backend# Error fetching models: HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url: /api/tags (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eeb2cdd5e10>: Failed to establish a new connection: [Errno 111] Connection refused'))
[Sun, 20 Jul 2025 08:40:02 GMT] "GET /health HTTP/1.1" 200 -
^C
root@952bd13803f1:/app/backend# exit
exit
oasis@wpnl:~/docker/localGPT$
Now, this works, which prompts my curiosity.
If it works manually as intended, then why is Docker failing the health check?
Does it require more delay time?
Why does the curl inside the container hang and not even show connection refused?
Is Ollama the cause of all of this?