[Bug]: [ERROR] memmachine.common.embedder.openai_embedder #918
Description
Describe the bug
As mentioned at #780, I have another issue when running memmachine-compose.sh with ollama embedder.
$ ./memmachine-compose.sh
...
Creating memmachine-postgres ... done
Creating memmachine-neo4j ... done
Creating memmachine-app ... done
[SUCCESS] Services started successfully!
[INFO] Waiting for services to be healthy...
Name Command State Ports
-------------------------------------------------------------------------------------------------------------------------
memmachine-app sh -c memmachine-server Up (health: starting) 0.0.0.0:8080->8080/tcp,:::8080->8080/tcp
memmachine-neo4j tini -g -- /startup/docker ... Up (healthy) 0.0.0.0:7473->7473/tcp,:::7473->7473/tcp,
0.0.0.0:7474->7474/tcp,:::7474->7474/tcp,
0.0.0.0:7687->7687/tcp,:::7687->7687/tcp
memmachine-postgres docker-entrypoint.sh postgres Up (healthy) 0.0.0.0:5432->5432/tcp,:::5432->5432/tcp
[INFO] Checking service health...
[INFO] Waiting for PostgreSQL to be ready...
/var/run/postgresql:5432 - accepting connections
[SUCCESS] PostgreSQL is ready
[INFO] Waiting for Neo4j to be ready...
[SUCCESS] Neo4j is ready
[INFO] Waiting for MemMachine to be ready...
[ERROR] MemMachine failed to become ready in 120 seconds. Check container logs and configuration.
Steps to reproduce
Use default config options except for using Ollama provider and the default nomic-embed-text, which is provided by ollama.
$ ./memmachine-compose.sh
MemMachine Docker Startup Script
====================================
[SUCCESS] Docker and Docker Compose are available
[WARNING] .env file not found. Creating from template...
[SUCCESS] Created .env file from sample_configs/env.dockercompose
[WARNING] configuration.yml file not found. Creating from template...
[PROMPT] Which configuration would you like to use for the Docker Image? (CPU/GPU) [CPU]:
[INFO] CPU configuration selected.
[PROMPT] Which provider would you like to use? (OpenAI/Bedrock/Ollama/OpenAI-compatible) [OpenAI]: Ollama
[INFO] Selected provider: OLLAMA
[SUCCESS] Set MEMMACHINE_IMAGE to memmachine/memmachine:latest-cpu in .env file
[PROMPT] Which Ollama LLM model would you like to use? [llama3]:
[SUCCESS] Selected Ollama LLM model: llama3
[PROMPT] Which Ollama embedding model would you like to use? [nomic-embed-text]:
[SUCCESS] Selected Ollama embedding model: nomic-embed-text
[INFO] Generating configuration file for OLLAMA provider...
[SUCCESS] Generated configuration file with OLLAMA provider settings
[PROMPT] Ollama base URL [http://host.docker.internal:11434/v1]:
[SUCCESS] Set Ollama base URL: http://host.docker.internal:11434/v1
[SUCCESS] Ollama configuration detected with default base URL
[SUCCESS] API key in configuration.yml appears to be configured
[SUCCESS] Database credentials in configuration.yml appear to be configured
[INFO] Pulling and starting MemMachine services...
[INFO] Pulling latest images... (Target: memmachine/memmachine:latest-cpu)
...
Expected behavior
MemMachine is ready.
Environment
- OS: Ubuntu 24.04 LTS
- MemMachine version: cb76064 Bump tzdata from 2025.2 to 2025.3 (Bump tzdata from 2025.2 to 2025.3 #893)
- Python version: Python 3.12.3
- Docker version 28.2.2
- docker-compose version 1.29.2
Additional context
Here is the log output.
$ ./memmachine-compose.sh logs
[INFO] Showing MemMachine logs...
Attaching to memmachine-app, memmachine-neo4j, memmachine-postgres
memmachine-app | 2026-01-07 22:17:06,263 [INFO] memmachine.server.api_v2.mcp - Configuration file 'configuration.yml' loaded.
memmachine-app | /app/.venv/lib/python3.12/site-packages/websockets/legacy/__init__.py:6: DeprecationWarning: websockets.legacy is deprecated; see https://websockets.readthedocs.io/en/stable/howto/upgrade.html for upgrade instructions
memmachine-app | warnings.warn( # deprecated in 14.0 - 2024-11-09
memmachine-app | /app/.venv/lib/python3.12/site-packages/uvicorn/protocols/websockets/websockets_impl.py:17: DeprecationWarning: websockets.server.WebSocketServerProtocol is deprecated
memmachine-app | from websockets.server import WebSocketServerProtocol
memmachine-app | INFO: Started server process [7]
memmachine-app | INFO: Waiting for application startup.
memmachine-app | 2026-01-07 22:17:06,325 [INFO] memmachine.common.configuration - Loading configuration from 'configuration.yml'
memmachine-app | 2026-01-07 22:17:06,330 [INFO] memmachine.common.configuration.log_conf - applying log configuration: level=INFO, format=%(asctime)s [%(levelname)s] %(name)s - %(message)s, path=mem-machine.log
memmachine-app | 2026-01-07 22:17:06,330 [INFO] memmachine.server.api_v2.mcp - Configuration file 'configuration.yml' loaded.
memmachine-app | 2026-01-07 22:17:06,330 [INFO] memmachine.common.configuration.log_conf - applying log configuration: level=INFO, format=%(asctime)s [%(levelname)s] %(name)s - %(message)s, path=mem-machine.log
memmachine-app | 2026-01-07 22:17:06,520 [INFO] alembic.runtime.migration - Context impl PostgresqlImpl.
memmachine-app | 2026-01-07 22:17:06,521 [INFO] alembic.runtime.migration - Will assume transactional DDL.
memmachine-app | 2026-01-07 22:17:06,528 [INFO] alembic.runtime.migration - Running upgrade -> 001, Initial migration - original schema without SQLAlchemy.
memmachine-app | 2026-01-07 22:17:06,542 [INFO] alembic.runtime.migration - Running upgrade 001 -> 3d6aaebdc526, sync_sqlalchemy_schema.
memmachine-app | 2026-01-07 22:17:06,571 [INFO] alembic.runtime.migration - Running upgrade 3d6aaebdc526 -> 843f6d216d10, Rename semantic type to semantic category.
memmachine-app | 2026-01-07 22:17:06,587 [INFO] alembic.runtime.migration - Running upgrade 843f6d216d10 -> adb5618bd4ee, Remove history table.
memmachine-app | 2026-01-07 22:17:06,589 [INFO] alembic.runtime.migration - Running upgrade adb5618bd4ee -> 79f00a9f2409, Change history id to string.
memmachine-app | 2026-01-07 22:17:06,596 [INFO] alembic.runtime.migration - Running upgrade 79f00a9f2409 -> 62dff1150a46, Add created at to history add.
memmachine-app | 2026-01-07 22:17:06,648 [INFO] memmachine.common.resource_manager.embedder_manager - Validating embedder 'ollama_embedder' is working.
memmachine-app | 2026-01-07 22:17:06,992 [INFO] openai._base_client - Retrying request to /embeddings in 0.446959 seconds
memmachine-app | 2026-01-07 22:17:07,607 [INFO] openai._base_client - Retrying request to /embeddings in 0.947527 seconds
memmachine-app | 2026-01-07 22:17:08,834 [ERROR] memmachine.common.embedder.openai_embedder - [call uuid: d84778a3-7aa1-43f7-a74b-2cb57b541442] Giving up creating embeddings for cluster number 0 after failed attempt 1 due to retryable APIConn
ectionError: max attempts 1 reached
memmachine-app | Traceback (most recent call last):
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
memmachine-app | yield
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 394, in handle_async_request
memmachine-app | resp = await self._pool.handle_async_request(req)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request
memmachine-app | raise exc from None
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 236, in handle_async_request
memmachine-app | response = await connection.handle_async_request(
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request
memmachine-app | raise exc
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py", line 78, in handle_async_request
memmachine-app | stream = await self._connect(request)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py", line 124, in _connect
memmachine-app | stream = await self._network_backend.connect_tcp(**kwargs)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_backends/auto.py", line 31, in connect_tcp
memmachine-app | return await self._backend.connect_tcp(
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 113, in connect_tcp
memmachine-app | with map_exceptions(exc_map):
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/usr/local/lib/python3.12/contextlib.py", line 158, in __exit__
memmachine-app | self.gen.throw(value)
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
memmachine-app | raise to_exc(exc) from exc
memmachine-app | httpcore.ConnectError: [Errno -2] Name or service not known
memmachine-app |
memmachine-app | The above exception was the direct cause of the following exception:
memmachine-app |
memmachine-app | Traceback (most recent call last):
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1532, in request
memmachine-app | response = await self._client.send(
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1629, in send
memmachine-app | response = await self._send_handling_auth(
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1657, in _send_handling_auth
memmachine-app | response = await self._send_handling_redirects(
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects
memmachine-app | response = await self._send_single_request(request)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1730, in _send_single_request
memmachine-app | response = await transport.handle_async_request(request)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 393, in handle_async_request
memmachine-app | with map_httpcore_exceptions():
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/usr/local/lib/python3.12/contextlib.py", line 158, in __exit__
memmachine-app | self.gen.throw(value)
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
memmachine-app | raise mapped_exc(message) from exc
memmachine-app | httpx.ConnectError: [Errno -2] Name or service not known
memmachine-app |
memmachine-app | The above exception was the direct cause of the following exception:
memmachine-app |
memmachine-app | Traceback (most recent call last):
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/memmachine/common/embedder/openai_embedder.py", line 236, in _embed_chunk_cluster
memmachine-app | await self._client.embeddings.create(
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/openai/resources/embeddings.py", line 251, in create
memmachine-app | return await self._post(
memmachine-app | ^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1797, in post
memmachine-app | return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1564, in request
memmachine-app | raise APIConnectionError(request=request) from err
memmachine-app | openai.APIConnectionError: Connection error.
memmachine-app | ERROR: Traceback (most recent call last):
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
memmachine-app | yield
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 394, in handle_async_request
memmachine-app | resp = await self._pool.handle_async_request(req)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request
memmachine-app | raise exc from None
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 236, in handle_async_request
memmachine-app | response = await connection.handle_async_request(
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request
memmachine-app | raise exc
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py", line 78, in handle_async_request
memmachine-app | stream = await self._connect(request)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py", line 124, in _connect
memmachine-app | stream = await self._network_backend.connect_tcp(**kwargs)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_backends/auto.py", line 31, in connect_tcp
memmachine-app | return await self._backend.connect_tcp(
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 113, in connect_tcp
memmachine-app | with map_exceptions(exc_map):
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/usr/local/lib/python3.12/contextlib.py", line 158, in __exit__
memmachine-app | self.gen.throw(value)
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
memmachine-app | raise to_exc(exc) from exc
memmachine-app | httpcore.ConnectError: [Errno -2] Name or service not known
memmachine-app |
memmachine-app | The above exception was the direct cause of the following exception:
memmachine-app |
memmachine-app | Traceback (most recent call last):
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1532, in request
memmachine-app | response = await self._client.send(
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1629, in send
memmachine-app | response = await self._send_handling_auth(
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1657, in _send_handling_auth
memmachine-app | response = await self._send_handling_redirects(
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects
memmachine-app | response = await self._send_single_request(request)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1730, in _send_single_request
memmachine-app | response = await transport.handle_async_request(request)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 393, in handle_async_request
memmachine-app | with map_httpcore_exceptions():
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/usr/local/lib/python3.12/contextlib.py", line 158, in __exit__
memmachine-app | self.gen.throw(value)
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
memmachine-app | raise mapped_exc(message) from exc
memmachine-app | httpx.ConnectError: [Errno -2] Name or service not known
memmachine-app |
memmachine-app | The above exception was the direct cause of the following exception:
memmachine-app |
memmachine-app | Traceback (most recent call last):
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/memmachine/common/embedder/openai_embedder.py", line 236, in _embed_chunk_cluster
memmachine-app | await self._client.embeddings.create(
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/openai/resources/embeddings.py", line 251, in create
memmachine-app | return await self._post(
memmachine-app | ^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1797, in post
memmachine-app | return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1564, in request
memmachine-app | raise APIConnectionError(request=request) from err
memmachine-app | openai.APIConnectionError: Connection error.
memmachine-app |
memmachine-app | The above exception was the direct cause of the following exception:
memmachine-app |
memmachine-app | Traceback (most recent call last):
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/memmachine/common/resource_manager/embedder_manager.py", line 64, in _validate_embedder
memmachine-app | _ = await embedder.search_embed(["a"])
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/memmachine/common/embedder/openai_embedder.py", line 126, in search_embed
memmachine-app | return await self._embed(queries, max_attempts)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/memmachine/common/embedder/openai_embedder.py", line 177, in _embed
memmachine-app | clusters_chunk_embeddings = await asyncio.gather(
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/memmachine/common/embedder/openai_embedder.py", line 273, in _embed_chunk_cluster
memmachine-app | raise ExternalServiceAPIError(error_message) from err
memmachine-app | memmachine.common.data_types.ExternalServiceAPIError: [call uuid: d84778a3-7aa1-43f7-a74b-2cb57b541442] Giving up creating embeddings for cluster number 0 after failed attempt 1 due to retryable APIConnectionError: max attemp
ts 1 reached
memmachine-app |
memmachine-app | The above exception was the direct cause of the following exception:
memmachine-app |
memmachine-app | Traceback (most recent call last):
memmachine-app | File "/app/.venv/lib/python3.12/site-packages/starlette/routing.py", line 694, in lifespan
memmachine-app | async with self.lifespan_context(app) as maybe_state:
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^^^^^^
memmachine-app | File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
memmachine-app | return await anext(self.gen)
memmachine-app | ^^^^^^^^^^^^^^^^^^^^^