{"id":703,"date":"2024-01-06T23:22:23","date_gmt":"2024-01-06T15:22:23","guid":{"rendered":"http:\/\/madapapa.com\/wordpress\/?p=703"},"modified":"2024-01-15T21:15:17","modified_gmt":"2024-01-15T13:15:17","slug":"shang-tang-dellm-shi-li-dai-ma-fen-xi","status":"publish","type":"post","link":"http:\/\/madapapa.com\/wordpress\/?p=703","title":{"rendered":"\u5546\u6c64\u7684llm\u793a\u4f8b\u4ee3\u7801\u5206\u6790"},"content":{"rendered":"

\u8fd9\u662f\u5546\u6c64sensennova\u5927\u8bed\u8a00\u6a21\u578b\u7684\u793a\u4f8b\u4ee3\u7801\uff08\u5bf9\u8bdd\u751f\u6210_\u65e0\u5bf9\u8bdd\u5386\u53f2\uff09<\/p>\n

stream = True # \u6d41\u5f0f\u8f93\u51fa\u6216\u975e\u6d41\u5f0f\u8f93\u51fa\nmodel_id = "nova-ptc-xl-v1" # \u586b\u5199\u771f\u5b9e\u7684\u6a21\u578bID\n\nresp = sensenova.ChatCompletion.create(\n    messages=[{"role": "user", "content": "\u4eba\u751f\u5929\u5730\u95f4\uff0c\u4e0b\u4e00\u53e5\u662f\u5565"}],\n    model=model_id,\n    stream=stream,\n    max_new_tokens=1024,\n    n=1,\n    repetition_penalty=1.05,\n    temperature=0.9,\n    top_p=0.7,\n    know_ids=[],\n    user="sensenova-python-test-user",\n    knowledge_config={\n        "control_level": "normal",\n        "knowledge_base_result": True,\n        "knowledge_base_configs":[]\n    },\n    plugins={\n        "associated_knowledge": {\n            "content": "\u9700\u8981\u6ce8\u5165\u7ed9\u6a21\u578b\u7684\u77e5\u8bc6",\n            "mode": "concatenate"\n        },\n        "web_search": {\n            "search_enable": True,\n            "result_enable": True\n        },\n    }\n)\n\nif not stream:\n    resp = [resp]\nfor part in resp:\n    choices = part['data']["choices"]\n    for c_idx, c in enumerate(choices):\n        if len(choices) > 1:\n            sys.stdout.write("===== Chat Completion {} =====\\n".format(c_idx))\n        if stream:\n            delta = c.get("delta")\n            if delta:\n                sys.stdout.write(delta)\n        else:\n            sys.stdout.write(c["message"])\n            if len(choices) > 1:\n                sys.stdout.write("\\n")\n        sys.stdout.flush()\n<\/code><\/pre>\n
\u4e0b\u9762\u6211\u4eec\u6765line by line\u7684\u89e3\u91ca<\/p>\n
<\/span><\/a>\u82f1\u6587<\/h3>\n
    \n
  1. stream = True: This line sets a variable named stream. If True, it indicates that the output will be in a streaming fashion. Streaming output is typically used for real-time data processing.<\/li>\n
  2. model_id = "nova-ptc-xl-v1": Here, model_id is set to the string "nova-ptc-xl-v1", which seems to be the identifier of a specific model in the Sensenova framework.<\/li>\n
  3. resp = sensenova.ChatCompletion.create(…): This line calls the create method of the ChatCompletion class from the sensenova module. This method likely generates a chat completion or response based on the given parameters.<\/li>\n
  4. messages=[{"role": "user", "content": "\u4eba\u751f\u5929\u5730\u95f4\uff0c\u4e0b\u4e00\u53e5\u662f\u5565"}]: Specifies the input message for the chat model. Here, the role is set as "user" and the content is a Chinese phrase.<\/li>\n
  5. model=model_id: Specifies the model to be used for chat completion.<\/li>\n
  6. stream=stream: Passes the stream variable to control streaming behavior.<\/li>\n
  7. max_new_tokens=1024: Limits the number of new tokens (words\/characters) that the model can generate.<\/li>\n
  8. n=1: Indicates that the number of completions to generate is 1.<\/li>\n
  9. repetition_penalty=1.05: Adjusts the likelihood of the model repeating the same line.<\/li>\n
  10. temperature=0.9: Sets the creativity or randomness of the response.<\/li>\n
  11. top_p=0.7: Controls the diversity of the response.<\/li>\n
  12. know_ids=[]: An empty list, possibly for specifying known IDs for context or tracking.<\/li>\n
  13. user="sensenova-python-test-user": Specifies a user identifier.<\/li>\n
  14. knowledge_config={…}: Configures how the model should use external knowledge.<\/li>\n
  15. plugins={…}: Configures additional features or plugins, like injecting knowledge (associated_knowledge) and enabling web search (web_search).<\/li>\n
  16. if not stream: resp = [resp]: If stream is False, this wraps resp in a list. This is probably for consistency in processing the response whether it’s streaming or not.<\/li>\n
  17. for part in resp:: Iterates over each part in the response.<\/li>\n
  18. choices = part[‘data’]["choices"]: Extracts the choices from the response data. This might contain the actual chat completions or responses.<\/li>\n
  19. for c_idx, c in enumerate(choices):: Iterates over the choices, with c_idx being the index and c being the content.<\/li>\n
  20. if len(choices) > 1: …: If there are multiple choices, it prints a separator with the index.<\/li>\n
  21. Inside the nested if and else:<\/li>\n
  22. If stream is True, it prints delta from each choice, which might be a part of the streaming response.<\/li>\n
  23. If stream is False, it prints the message from each choice.<\/li>\n
  24. sys.stdout.flush(): Ensures that everything written to the standard output is actually displayed. This is often used in streaming or real-time applications to update the output promptly.<\/li>\n<\/ol>\n
    This code seems to be a part of a chatbot implementation using the Sensenova framework, where it sends a message to a chat model and processes the response, either in a streaming manner or as a complete response.<\/p>\n
    <\/span><\/a>\u4e2d\u6587<\/h3>\n