weaviate连接缓慢原因分析

weaviate-client连接本地服务器缓慢原因分析

我们使用python的weaviate-client客户端连接本地的weaviate时，经常会很慢，具体的代码如下，使用下面方法，可以使你的weaviate连接时提速99%。

import weaviate
client = weaviate.Client(
    url = "http://127.0.0.0:8080",  # Replace with your endpoint
)

分析发现，服务器端没有看到任何建立连接的信息，所以这个原因就是在客户端这里。

通过追踪堆栈，发现如下：

  File "embedding_find.py", line 187, in <module>
    client = weaviate.Client(
             ^^^^^^^^^^^^^^^^
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/site-packages/weaviate/client.py", line 150, in __init__
    self._connection = Connection(
                       ^^^^^^^^^^^
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/site-packages/weaviate/connect/connection.py", line 667, in __init__
    pkg_info = requests.get(PYPI_PACKAGE_URL).json()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/site-packages/requests/adapters.py", line 487, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/site-packages/urllib3/connectionpool.py", line 714, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/site-packages/urllib3/connectionpool.py", line 403, in _make_request
    self._validate_conn(conn)
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1053, in _validate_conn
    conn.connect()
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/site-packages/urllib3/connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
                ^^^^^^^^^^^^^^^^
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/ssl.py", line 1075, in _create
    self.do_handshake()
  File "/Users/admin/miniforge3/envs/chat/lib/python3.11/ssl.py", line 1346, in do_handshake
    self._sslobj.do_handshake()
KeyboardInterrupt
sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=7, family=2, type=1, proto=0, laddr=('127.0.0.1', 59984), raddr=('151.101.64.223', 443)>

可以发现建立连接时，weaviate尝试连接PYPI_PACKAGE_URL，

通过查看源码我们发现PYPI_PACKAGE_URL = “https://pypi.org/pypi/weaviate-client/json"

原来缓慢的原因如下，因为pypi官网是在美国，中国由于GFW问题，导致访问pypi经常抽风，所以导致weaviate在检查版本信息时，连接不上，这导致了报错，我们可以注释掉这段检查的源码即可。

源码的位置在： site-packages/weaviate/connect/connection.py中的class Connection(BaseConnection):中的__init__函数中。

pkg_info = requests.get(PYPI_PACKAGE_URL).json()
pkg_info = pkg_info.get("info", {})
latest_version = pkg_info.get("version", "unknown version")
if is_weaviate_client_too_old(client_version, latest_version):
    _Warnings.weaviate_client_too_old_vs_latest(client_version, latest_version)