Category: LINUX

langchain应用课程

这是吴恩达和langchain创始人联合开的一门课，目前我在学两个课程，1个是应用框架介绍，1个是langchain和data结合。
langchain application course

##我的初步思考

  • langchain的应用框架，核心，是为了更好的完成对llm的输入和输出。为什么普通用户不可以完成一个完美的prompt，因为缺乏训练，或者因为完美的prompt重复性内容很多，特别是针对某一类垂直领域的业务，其模式是非常固定的，人就懒惰了。
    1. 我们首先要完成一个完美的prompt（和parsers） ；
    2. 然后分析这个prompt，里面有哪些内容是可以重复使用？灵活使用的？把这部分拿出来作为模版或者变量，例如：语言的选择，形式的选择，语气的选择等待；
    3. 在上面业务分析的基础上，利用langchain的各种组建，实现上面的设计。

chatapi有两种方式

直接调用openai的api

response = openai.ChatCompletion.create(

        model=model,

        messages=messages,

        temperature=0, 

    )

调用langchain来实现

chat = ChatOpenAI(temperature=0.0, model=llm_model)

维特根斯坦说过，你如何使用语言，你就如何思维；你如何思维，你也如何使用语言。
你如何使用语言呢，其实就是你语言的关键词有哪些，这些关键词的顺序是怎样的。

  • thought是llm应该如何去理解问题；
  • Action是llm应该采取什么行动；
  • Oberservation是llm从action中得到了什么；

参考-维特根斯坦说过的话

  1. "语言不仅仅是传达思想的工具；它也塑造和限制了我们所能思考的内容。" – 这句话源自维特根斯坦的著作《逻辑哲学论》(Tractatus Logico-Philosophicus)。维特根斯坦在这里阐述了一个关于语言和思维的基本观点：语言不仅仅是用来表达我们已有的思想，它实际上还在形塑和限制我们的思维方式。换句话说，我们能够想到什么，以及我们如何去思考，很大程度上受到我们所使用的语言结构和词汇的限制。例如，如果某种语言没有表达特定概念的词汇，那么使用这种语言的人可能很难理解或思考这个概念。这体现了语言不只是思维的载体，也是思维的塑造者和限制者。

  2. "我们的语言是我们思考的镜子。" – 来自《文化与价值》(Culture and Value)。在这句话中，维特根斯坦强调了语言和思维之间的反映和相互作用。这里，语言被视为一种反映我们思考方式的工具。就像镜子一样，语言展示了我们的思考模式和心理过程。这意味着通过观察和分析我们的语言使用，我们可以了解自己的思维习惯和方式。这种观点也强调了语言和思维之间不可分割的关系，表明我们的语言实际上揭示了我们的思考特征和倾向。

openai的sdk又升级了

 % pip3 install --upgrade openai

openai.ChatCompletion.create() -> client.chat.completions.create()

    response = openai.chat.completions.create(
        model = model,
        messages = messages,
        temperature = 0,
    )
    return response.choices[0].message.content

我实际没有修改client，也是成功的。但是从下面吴恩达的介绍看，response也要修改为属性调用，而不是直接调用。

不过经过chatgpt的解释，我还是使用了client这个实例来调用相关函数，原因如下（更灵活，更好支持异步操作、更符合行业惯例）

The shift from using openai.chat to client.chat in the OpenAI API reflects a change in the design of the OpenAI Python library. Here are the key reasons for this change:

Client Instance Configuration: 
Using a client instance allows for more flexible configuration. You can create multiple client instances with different configurations (e.g., different API keys, different base URLs for testing or production environments) within the same application. This is more scalable and adaptable for complex applications or those that need to interact with multiple environments or configurations.

Asynchronous Support: 
The new API design is likely structured to better support asynchronous operations. By using a client object, it's easier to manage asynchronous requests and responses, which are becoming increasingly important for efficient handling of network operations, especially in web applications and services.

Consistency with Other SDKs: 
Many modern API SDKs (Software Development Kits) use a similar pattern where you create a client instance and then use methods on this instance to interact with the API. This design pattern is familiar to many developers and aligns with best practices in software development.

Encapsulation and Extensibility: 
By using a client object, the OpenAI library can encapsulate functionality and state more effectively. This makes the library easier to extend and maintain. It also allows for better handling of resources like network connections.

Error Handling and Debugging: 
A client-based approach can offer improved error handling and debugging capabilities. Since each client instance can manage its own state and configuration, it's easier to trace and handle errors, log activities, and debug issues.

In summary, the shift to using a client instance in the OpenAI Python library is likely driven by a desire for greater flexibility, support for asynchronous operations, consistency with modern software development practices, improved encapsulation and extensibility, and enhanced error handling and debugging capabilities.

更详细的异步调用的好处

The emphasis on asynchronous support in the new API design, particularly with the use of a client object, reflects a broader trend in software development towards more efficient and scalable network operations. Here's a deeper look into why this is beneficial:

1. Improved Performance and Scalability:
Non-blocking Calls: Asynchronous operations allow your application to make non-blocking network calls. This means that your application doesn't have to wait for a response from the OpenAI server before continuing with other tasks. This is especially beneficial in web applications where multiple users might be making requests at the same time.
Handling Multiple Requests: Asynchronous programming is more efficient at handling multiple simultaneous network requests. This is crucial for high-load applications that need to maintain responsiveness under heavy request volumes.

2. Better Resource Utilization:
Concurrency: Asynchronous operations enable better utilization of system resources. While waiting for a response from an API call, your application can perform other tasks, thereby making better use of the CPU and other resources.
Reduced Latency: In a synchronous model, each operation must complete before the next one starts, potentially leading to higher latency. Asynchronous operations can overlap, which can reduce overall latency in the application.

3. Enhanced User Experience:
Responsive Applications: In a web or mobile application, asynchronous operations can significantly enhance the user experience. Users aren't left waiting for operations to complete and can continue interacting with other parts of the application.
Real-time Updates: Asynchronous programming facilitates real-time updates to the user interface, which can be crucial for applications that require immediate feedback, such as chatbots or live data monitoring.

4. Simplified Error Handling:
Asynchronous workflows: Asynchronous programming often comes with more sophisticated ways to handle errors and exceptions. For instance, in Python's asyncio, you can use try/except blocks within asynchronous functions to manage exceptions effectively.

5. Alignment with Modern Web Standards:
WebSockets and HTTP/2: Modern web protocols like WebSockets and HTTP/2 are designed to work efficiently with asynchronous communication, making it a natural fit for applications that leverage these technologies.

Implementation in Python:
asyncio Library: Python’s asyncio library is a popular choice for writing asynchronous code. It provides a framework for writing single-threaded concurrent code using coroutines, multiplexing I/O access, and running network clients and servers.
Integration with Frameworks: Many modern Python web frameworks (like FastAPI, Sanic, etc.) are built with native support for asynchronous operations, making it easier to integrate with asynchronous APIs like OpenAI's.

In summary, the shift towards asynchronous support with a client object in API design is a response to the growing need for more efficient, scalable, and responsive applications, particularly in the context of web and network services. It aligns well with modern software practices and technological advancements.

参考这篇文档

我的mac terminal electerm

我一直在用的mac终端是electerm，不是很有名，我甚至想不起它的名字，不过因为这款软件支持ssh、ftp，虽然有时候速度比较慢，但是我很喜欢。
支持mac、linux和windows

安装很简单

brew install --cask electerm

介绍很简洁

github官网
https://github.com/electerm/electerm
知乎介绍
https://zhuanlan.zhihu.com/p/348324919

intel网卡安装奇遇记

最近项目上碰到一个有意思的事，客户要求实现roce组网，但是云服务商提供的网卡是intel800系列，驱动没有适配国产OS。

官网产品主页

intel官方驱动只支持红帽和suse

从BMC查询的网卡型号为NIC-E810CQDA2G2P5-E810，在intel官网上查询（第二次查询才找到这个全量包），发现其对os的支持范围非常窄（红帽、SUSE、ubuntu等），没有对国产麒麟、欧拉的支持。

第一次查询E810网卡驱动时，base driver,rdma driver都是单独的页面

真心话，intel的驱动查询网站着实不如英伟达的

驱动和操作系统适配


从上表可以看出，810系列网卡，其驱动实际分为两种，base driver和rdma driver,其中base driver又分为物理机和虚拟机两种驱动。

安装驱动

intel的驱动安装相比英伟达网卡，要复杂一些。首先，要安装网卡的基本驱动，启动网卡并设置ip地址。然后再安装rdma驱动，并安装rdma库（从github下载）。最后，为了最大化rdma的性能，要打开流量控制标识（PFC或LFC），同时对应交换机的端口也要设置PFC。

RDMA协议里，只有rocev2从pfc收益最多，原因也很简单，roce是over udp的，而udp可以说是原本就不靠谱的以太网协议里，更不靠谱的一种协议，如果没有流量控制，当出现丢包时，整个网络传输就基本不可用了。

供应商

服务器硬件供应商提供了一个网卡驱动“nic-e810-kylinlinuxv10sp2.rpm，吊诡的是，他们找不到 下载链接。

驱动下载

除了github外，20年前最火爆的开源项目网站是sourceforge.
目前sourceforge活得有滋有味，因为不可言说的原因，在国内可以畅通无阻访问，这点完胜github。

总结，以后配套设备驱动下载的顺序依次为

  • 原厂商官网，如intel，英伟达
  • 集成商或经销商的官网，一般可以在原厂商官网上找到，如intel
  • 开源网站

更高级的minikube命令

几个高级的minikube命令

Now that we have familiarized ourselves with the default minikube start command, let’s dive deeper into Minikube to understand some of its more advanced features.

The minikube start by default selects a driver isolation software, such as a hypervisor or a container runtime, if one (VitualBox) or multiple are installed on the host workstation. In addition it downloads the latest Kubernetes version components. With the selected driver software it provisions a single VM named minikube (with hardware profile of CPUs=2, Memory=6GB, Disk=20GB) or container to host the default single-node all-in-one Kubernetes cluster. Once the node is provisioned, it bootstraps the Kubernetes control plane (with the default kubeadm tool), and it installs the latest version of the default container runtime, Docker, that will serve as a running environment for the containerized applications we will deploy to the Kubernetes cluster. The minikube start command generates a default minikube cluster with the specifications described above and it will store these specs so that we can restart the default cluster whenever desired. The object that stores the specifications of our cluster is called a profile.

As Minikube matures, so do its features and capabilities. With the introduction of profiles, Minikube allows users to create custom reusable clusters that can all be managed from a single command line client.

The minikube profile command allows us to view the status of all our clusters in a table formatted output. Assuming we have created only the default minikube cluster, we could list the properties that define the default profile with:

$ minikube profile list

|———-|————|———|—————-|——|———|———|——-|——–|

Profile VM Driver Runtime IP Port Version Status Nodes Active
minikube virtualbox docker 192.168.59.100 8443 v1.25.3 Running 1 *
———- ———— ——— —————- —— ——— ——— ——- ——–

This table presents the columns associated with the default properties such as the profile name: minikube, the isolation driver: VirtualBox, the container runtime: Docker, the Kubernetes version: v1.25.3, the status of the cluster – running or stopped. The table also displays the number of nodes: 1 by default, the private IP address of the minikube cluster’s control plane VirtualBox VM, and the secure port that exposes the API Server to cluster control plane components, agents and clients: 8443.

What if we desire to create several reusable clusters instead, with other drivers (Docker or Podman (still experimental on Linux)) for node isolation, or different Kubernetes versions (v1.23.3 or v1.24.4), another runtime (cri-o or containerd), and possibly 2, 3, or more nodes (if permitted by the resources of our host system)? What if we desire to further customize the cluster with a specific networking option or plugin? The minikube start command allows us to create such custom profiles with the –profile or -p flags. Several of the isolation drivers support creation of node VMs or node containers of custom sizes as well, features that we will not explore in this course as not all are very stable at the time of this writing.

Below are a few examples of more complex start commands that allow custom clusters to be created with Minikube. They assume that the desired driver software (Docker and/or Podman) has been installed on the host workstation. There is no need to download the desired CNI (network plugin) or the container runtime, they will be set up and enabled by Minikube on our behalf:

$ minikube start –kubernetes-version=v1.23.3
–driver=podman –profile minipod

$ minikube start –nodes=2 –kubernetes-version=v1.24.4
–driver=docker –profile doubledocker

$ minikube start –driver=virtualbox –nodes=3 –disk-size=10g
–cpus=2 –memory=4g –kubernetes-version=v1.25.1 –cni=calico
–container-runtime=cri-o -p multivbox

$ minikube start –driver=docker –cpus=6 –memory=8g
–kubernetes-version="1.24.4" -p largedock

$ minikube start –driver=virtualbox -n 3 –container-runtime=containerd
–cni=calico -p minibox

Once multiple cluster profiles are available (the default minikube and custom minibox), the profiles table will look like this:

$ minikube profile list

|———-|————|———|—————-|——|———|———|——-|——–|

Profile VM Driver Runtime IP Port Version Status Nodes Active
minibox virtualbox crio 192.168.59.101 8443 v1.25.3 Running 3
minikube virtualbox docker 192.168.59.100 8443 v1.25.3 Running 1 *
———- ———— ——— —————- —— ——— ——— ——- ——–

The active marker indicates the target cluster profile of the minikube command line tool. The target cluster can be set to minibox with the following command:

$ minikube profile minibox

The target cluster can be set to the default minikube with one of the following commands:

$ minikube profile minikube

$ minikube profile default

一个实际的例子

增加一个新的cluster

$ minikube start --driver=docker -p minibox

k8s亲和度调度

阿里云的这篇文章，介绍了如何利用亲和度（or反亲和度）特性，来实现pod的调度。
这里有一个例子，用来把pod调度到特定名字的node上。

比如要将Pod调度到nodeName是k8s-worker-2的节点上
apiVersion: v1
kind: Pod
metadata:
  name: webapp
  namespace: demo
  labels:
    app: webapp
spec:
  nodeName: 'k8s-worker-2'
  containers:
    - name: webapp
      image: nginx
      ports:
        - containerPort: 80

下面是chatgpt对这段脚本的详细解释

The YAML configuration you've provided is for a Kubernetes Pod named "webapp" in the "demo" namespace. Here's a breakdown of the configuration:

- `apiVersion: v1` and `kind: Pod`: These fields specify the Kubernetes API version and the resource type, which, in this case, is a Pod.

- `metadata`: This section contains metadata information for the Pod, such as its name, namespace, and labels.

  - `name: webapp`: The name of the Pod is set to "webapp."

  - `namespace: demo`: The Pod is created within the "demo" namespace.

  - `labels`: Labels are key-value pairs that help you organize and categorize Pods. In this case, the label "app" is set to "webapp."

- `spec`: This section defines the specifications of the Pod, including its containers, resources, and other settings.

  - `nodeName: 'k8s-worker-2'`: This field specifies the node (Kubernetes worker node) where the Pod should be scheduled. It pins the Pod to a specific node named "k8s-worker-2." This is known as node affinity and ensures that the Pod runs on a specific node, which can be useful for various scenarios such as hardware requirements or node-specific configurations.

  - `containers`: This is a list of containers running within the Pod. In this case, there is one container defined.

    - `name: webapp`: The container is named "webapp."

    - `image: nginx`: This field specifies the Docker image to use for this container. In this case, it uses the official Nginx image.

    - `ports`: This field defines the ports that are exposed by the container.

      - `containerPort: 80`: The container exposes port 80, which is the default port for HTTP traffic, allowing the Nginx web server within the container to listen for incoming HTTP requests.

This configuration will create a Pod running an Nginx web server on a specific node named "k8s-worker-2" in the "demo" namespace. The web server within the Pod will be accessible on port 80, and it has the label "app=webapp" for categorization and selection purposes.

kubelet and kubectl

Kubelet and kubectl are two essential components of the Kubernetes ecosystem, each serving distinct but interconnected roles.

Kubelet:

  • Definition:

    • Kubelet is an agent that runs on each node in a Kubernetes cluster. Its primary responsibility is to ensure that the containers running on the node are in the desired state, as defined by the Kubernetes control plane.

  • Key Functions:

    • Pod Lifecycle Management: Kubelet is responsible for starting, stopping, and maintaining the lifecycle of pods on its node. It communicates with the container runtime (e.g., Docker, containerd) to create and manage containers within pods.
    • Health Checks: Kubelet regularly performs health checks on the containers it manages. If a container or pod is not healthy, Kubelet can take corrective actions, such as restarting the container.
    • Resource Management: Kubelet ensures that containers have access to the necessary resources (CPU, memory) as specified in pod resource requests and limits.
    • Image Management: Kubelet pulls container images as needed and ensures they are available for pod instantiation.
    • Node Status: Kubelet reports the node’s status and resource usage to the Kubernetes control plane.

kubectl:

  • Definition:

    • kubectl (Kube Control) is the command-line tool used to interact with a Kubernetes cluster. It serves as the primary interface for administrators, developers, and operators to manage and control the cluster.

  • Key Functions:

    • Cluster Management: kubectl allows you to create, update, and delete Kubernetes resources like pods, services, deployments, and config maps.
    • Resource Inspection: You can use kubectl to inspect the status and details of resources in the cluster, e.g., kubectl get pods, kubectl describe service.
    • Scaling: kubectl enables you to scale deployments or replica sets up or down, e.g., kubectl scale deployment my-app –replicas=3.
    • Resource Creation: You can create resources from YAML or JSON files using kubectl apply -f <file.yaml>.
    • Pod Execution: You can execute commands inside pods using kubectl exec, access logs with kubectl logs, and copy files in and out of pods with kubectl cp.
    • Interactivity: kubectl offers an interactive mode, where you can run a shell in a container using kubectl exec -it.

In summary, Kubelet is an agent running on each node that takes care of pod and container management on that node, while kubectl is the command-line tool used to manage and control the entire Kubernetes cluster from a centralized location. They work together to ensure that containerized workloads are orchestrated and maintained according to the desired configuration.

flask的开发模式和生产模式（WSGI）

Flask容器化部署这篇文章介绍了一种利用WSGI技术，实现生产部署的方法。
WSGI介绍

我想了解flask run，下面是chatgpt的答复：

Flask is a micro web framework for Python that allows you to build web applications easily. When you use flask run to set up a web service, you are starting a development web server that comes bundled with Flask. This server is typically used for development and debugging purposes. It is not suitable for production use because it is single-threaded, not very robust, and not designed to handle a high volume of requests.

On the other hand, when you use a WSGI (Web Server Gateway Interface) server to deploy your Flask application, you are using a production-ready server to serve your application. WSGI is a standardized interface that allows web applications written in Python, like Flask, to be deployed on various web servers. When you deploy your Flask application with a WSGI server, such as Gunicorn, uWSGI, or mod_wsgi, you get a more robust and scalable web server setup that can handle concurrent requests and is better suited for production use.

Here are some key differences and considerations:

  1. Development vs. Production: flask run is intended for development and debugging, while WSGI servers are used in production environments.

  2. Scalability: WSGI servers can handle multiple concurrent requests and are suitable for scaling your application as traffic increases. Flask’s development server is not optimized for high concurrency.

  3. Reliability: WSGI servers are more reliable and have features like process management, worker processes, and failover mechanisms, making them better suited for handling production workloads.

  4. Deployment: When deploying your Flask application in a production environment, it’s recommended to use a WSGI server in conjunction with a reverse proxy server (like Nginx or Apache) for better performance, security, and load balancing.

In summary, while flask run is convenient for quickly testing your Flask application during development, it’s not suitable for production use. When deploying your Flask application for production, you should use a WSGI server for better performance, concurrency, and reliability.

在宿主机和容器里，分别查看ip地址

在mac上查看ip地址，比较方便的命令是

ifconfig en0 | grep inet

这条命令的解释如下

  1. Executes the "ifconfig" command to retrieve network configuration information for the "en0" network interface.
  2. Pipes (|) the output of the "ifconfig" command to the "grep inet" command.
  3. The "grep" command searches through the output and extracts any lines that contain the word "inet."

输出如下图

另外，也可以使用下面的命令查看ip地址，可以发现在宿主机和容器里，显示的ip地址是相同的。

# curl ifconfig.me
1.193.44.59#

宿主机

容器

解释如下

  • The command "curl ifconfig.me" is used to make an HTTP request to the ifconfig.me website, which is a service that provides your public IP address.

  • In the output you provided, "1.193.44.59" is the public IP address that was returned by the ifconfig.me service in response to your request. This is the public IP address that your device is using to communicate with the internet. It’s the address that other devices on the internet will see when they interact with your device.

  • Please note that public IP addresses can change, especially if you are using a dynamic IP assignment from your Internet Service Provider (ISP), so the IP address you see here may not be constant and could change over time.

在容器的bash环境里，输入hostname -i，显示当前容器对外暴露的ip地址

# hostname -i
XXX:XXXX:XXXa::d 192.168.194.13

这条命令解释如下

  • The command "hostname -i" is used to display the IP addresses associated with the current host or system. In the output you provided, you have two IP addresses listed:

    1. d07:b51a:cc66:a::d: This is an IPv6 address. IPv6 is the next-generation Internet Protocol designed to replace IPv4 due to the exhaustion of available IPv4 addresses.
    2. 192.168.194.13: This is an IPv4 address. IPv4 is the older and more widely used version of the Internet Protocol.

  • The output shows that your system has both an IPv6 and an IPv4 address associated with it. These addresses can be used for communication with other devices on a network, with IPv6 being used to support the transition to a more extensive address space as IPv4 addresses become scarcer.

查看docker 容器状态

有时候需要查看宿主机上的docker服务状态，以及哪些容器在运行或情况如何

查看docker服务状态

systemctl is-active docker



systemctl status docker

systemctl status [service name]命令会持续、实时显示相关服务的状态信息，如果要退出systemctl status状态回到提示符，可以：

Press Q: You can simply press the "Q" key on your keyboard to quit the "systemctl status" display. This will return you to the command line.

Press Ctrl + C: You can also press "Ctrl" and "C" simultaneously to send an interrupt signal to the "systemctl status" command, which will exit it and return you to the command line.

查看运行的容器状态

docker ps

查看所有容器状态

docker ps -a