sentence_transformers联网报错问题

国内由于墙的问题,经常会连接huggingface报错

类似报错如下:
curl: (28) Failed to connect to huggingface.co port 443: Connection timed out

测试本地网络:
curl https://huggingface.co/api/models/GanymedeNil/text2vec-base-chinese
curl: (28) Failed to connect to huggingface.co port 443: Connection timed out

确实不同,通过代理访问, 能正常返回

1
{"_id":"6406b3d583b59abcddefab03","id":"GanymedeNil/text2vec-base-chinese","modelId":"GanymedeNil/text2vec-base-chinese","author":"GanymedeNil","sha":"63c5640935ea81c04763e9dc3af5a731d93e16bf","lastModified":"2023-03-07T04:07:44.000Z","private":false,"disabled":false,"gated":false,"pipeline_tag":"sentence-similarity","tags":["pytorch","bert","feature-extraction","zh","transformers","text2vec","sentence-similarity","license:apache-2.0","has_space"],"downloads":15018,"library_name":"transformers","mask_token":"[MASK]","widgetData":[{"source_sentence":"那是 個快樂的人","sentences":["那是 條快樂的狗","那是 個非常幸福的人","今天是晴天"]}],"likes":6,"model-index":null,"config":{"architectures":["BertModel"],"model_type":"bert"},"cardData":{"license":"apache-2.0","language":["zh"],"pipeline_tag":"sentence-similarity","tags":["text2vec","feature-extraction","sentence-similarity","transformers"]},"transformersInfo":{"auto_model":"AutoModel","pipeline_tag":"feature-extraction","processor":"AutoTokenizer"},"spaces":["thomas-yanxin/LangChain-ChatLLM","Nicholaspei/LangChain-ChatLLM","ytjoh/LangChain-ChatLLM","chow-q/LangChain-ChatLLM","clc007/LangChain-ChatLLM"],"siblings":[{"rfilename":".gitattributes"},{"rfilename":"README.md"},{"rfilename":"config.json"},{"rfilename":"eval_results.txt"},{"rfilename":"pytorch_model.bin"},{"rfilename":"special_tokens_map.json"},{"rfilename":"tokenizer.json"},{"rfilename":"tokenizer_config.json"},{"rfilename":"vocab.txt"}]}

解决办法

从其它地方拷贝模型到本地,然后model_name改成本地的模型文件夹位置,不让模型联网进行检查

1
2
3
4
import sentence_transformers
self.client = sentence_transformers.SentenceTransformer(
model_name, cache_folder=self.cache_folder, **self.model_kwargs
)

sentence_transformers联网报错问题
https://johnson7788.github.io/2023/05/08/sentence-transformers%E8%81%94%E7%BD%91%E6%8A%A5%E9%94%99%E9%97%AE%E9%A2%98/
作者
Johnson
发布于
2023年5月8日
许可协议