Skip to content

Commit 8623edd

Browse files
author
will.yang
committed
release v1.1.4
1 parent 1a502af commit 8623edd

File tree

523 files changed

+545603
-31
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

523 files changed

+545603
-31
lines changed

README.md

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,8 @@
3030
- [x] [InternLM2 models](https://huggingface.co/collections/internlm/internlm2-65b0ce04970888799707893c)
3131
- [x] [MiniCPM models](https://huggingface.co/collections/openbmb/minicpm-65d48bf958302b9fd25b698f)
3232
- [x] [TeleChat models](https://huggingface.co/Tele-AI)
33+
- [x] [Qwen2-VL](https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct)
34+
- [x] [MiniCPM-V](https://huggingface.co/openbmb/MiniCPM-V-2_6)
3335

3436
# Model Performance Benchmark
3537

@@ -75,6 +77,12 @@
7577

7678
You can download the latest package, docker image, example, documentation, and platform-tool from [RKLLM_SDK](https://console.zbox.filez.com/l/RJJDmB), fetch code: rkllm
7779

80+
# Examples
81+
82+
1. Multimodel deployment demo: [rkllm_multimodel_demo](https://github.com/airockchip/rknn-llm/tree/main/examples/rkllm_multimodel_demo)
83+
2. API usage demo: [rkllm_api_demo](https://github.com/airockchip/rknn-llm/tree/main/examples/rkllm_api_demo)
84+
3. API server demo: [rkllm_server_demo](https://github.com/airockchip/rknn-llm/tree/main/examples/rkllm_server_demo)
85+
7886
# Note
7987

8088
- The modifications in version 1.1 are significant, making it incompatible with older version models. Please use the latest toolchain for model conversion and inference.
@@ -85,7 +93,7 @@ You can download the latest package, docker image, example, documentation, and p
8593

8694
- Python 3.10
8795

88-
- Latest version: [ <u>v1.1.3](https://github.com/airockchip/rknn-llm/releases/tag/release-v1.1.3)</u>
96+
- Latest version: [ <u>v1.1.4](https://github.com/airockchip/rknn-llm/releases/tag/release-v1.1.4)</u>
8997

9098
# RKNN Toolkit2
9199

@@ -109,4 +117,4 @@ https://github.com/airockchip/rknn-toolkit2
109117
- Add support for models such as Llama3, Gemma2, and MiniCPM3.
110118
- Resolve catastrophic forgetting issue when the number of tokens exceeds max_context.
111119

112-
for older version, please refer [CHANGELOG](CHANGELOG.md)
120+
for older version, please refer [CHANGELOG](CHANGELOG.md)

rkllm-runtime/examples/rkllm_api_demo/CMakeLists.txt renamed to examples/rkllm_api_demo/CMakeLists.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ add_executable(llm_demo ${SOURCE_FILES_1})
1010
set(SOURCE_FILES_2 src/multimodel_demo.cpp)
1111
add_executable(multimodel_demo ${SOURCE_FILES_2})
1212

13-
set(RKLLM_API_PATH "${CMAKE_SOURCE_DIR}/../../runtime/${CMAKE_SYSTEM_NAME}/librkllm_api")
13+
set(RKLLM_API_PATH "${CMAKE_SOURCE_DIR}/../../rkllm-runtime/${CMAKE_SYSTEM_NAME}/librkllm_api")
1414
include_directories(${RKLLM_API_PATH}/include)
1515
if(CMAKE_SYSTEM_NAME STREQUAL "Android")
1616
set(RKLLM_RT_LIB ${RKLLM_API_PATH}/${CMAKE_ANDROID_ARCH_ABI}/librkllmrt.so)
File renamed without changes.

rkllm-runtime/examples/rkllm_api_demo/build-android.sh renamed to examples/rkllm_api_demo/build-android.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ if [[ -z ${BUILD_TYPE} ]];then
44
BUILD_TYPE=Release
55
fi
66

7-
ANDROID_NDK_PATH=~/android-ndk-r21e
7+
ANDROID_NDK_PATH=~/opts/ndk/android-ndk-r21e
88
TARGET_ARCH=arm64-v8a
99

1010
TARGET_PLATFORM=android

rkllm-runtime/examples/rkllm_api_demo/build-linux.sh renamed to examples/rkllm_api_demo/build-linux.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ if [[ -z ${BUILD_TYPE} ]];then
44
BUILD_TYPE=Release
55
fi
66

7-
GCC_COMPILER_PATH=~/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu
7+
GCC_COMPILER_PATH=~/opts/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu
88
C_COMPILER=${GCC_COMPILER_PATH}-gcc
99
CXX_COMPILER=${GCC_COMPILER_PATH}-g++
1010
STRIP_COMPILER=${GCC_COMPILER_PATH}-strip
File renamed without changes.

rkllm-runtime/examples/rkllm_api_demo/src/multimodel_demo.cpp renamed to examples/rkllm_api_demo/src/multimodel_demo.cpp

File renamed without changes.
Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# Qwen2-VL-2B Demo
2+
1. This demo demonstrates how to deploy the Qwen2-VL-2B model. The Vision + Projector component is exported as an RKNN model using the `rknn-toolkit2`, while the LLM component is exported as an RKLLM model using the `rkllm-toolkit`.
3+
2. The open-source model used in this demo is available at: [Qwen2-VL-2B](https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct)
4+
5+
## 1. Requirements
6+
```
7+
rkllm-toolkit==1.1.4
8+
rknn-toolkit2==2.2.1
9+
python==3.8
10+
```
11+
12+
rknn-toolkit2 installation guide:
13+
14+
pip install rknn-toolkit2==2.2.1 -i https://mirrors.aliyun.com/pypi/simple
15+
16+
## 2. HuggingFace Demo
17+
18+
```
19+
1、modify the modelpath in infer.py
20+
2、python infer.py
21+
3、expect results:
22+
["The image depicts an astronaut in a white spacesuit, reclining on a green chair with his feet up. He is holding a green beer bottle in his right hand. The astronaut is on a moon-like surface, with the Earth visible in the background. The scene is set against a backdrop of stars and the moon's surface, creating a surreal and whimsical atmosphere."]
23+
```
24+
25+
## 3. Model Conversion
26+
- ### convert to onnx
27+
28+
1. Export the Vision + Projector component of the Qwen2-VL-2B model to an ONNX model using the `export/export_vision.py` script.
29+
30+
2. Since RKNN currently supports only `float32`, if the data type is restricted when loading weights, you need to set the `"use_flash_attn"` parameter in `config.json` to `false`.
31+
32+
```bash
33+
python export/export_vision.py
34+
```
35+
36+
- ### convert to rknn
37+
38+
1. After successfully exporting the ONNX model, you can use the `export/export_vision_rknn.py` script along with the `rknn-toolkit2` tool to convert the ONNX model to an RKNN model.
39+
40+
```bash
41+
python export/export_vision_rknn.py
42+
```
43+
44+
- ### convert to rkllm
45+
46+
1. We collected 20 image-text examples from the MMBench_DEV_EN dataset, stored in `data/datasets.json` and `data/datasets`. To use these data, you first need to create `input_embeds` for quantizing the RKLLM model. Run the following code to generate `data/inputs.json`.
47+
48+
```bash
49+
#Modify the Qwen2VL ModelPath in data/make_input_embeds_for_quantize.py, and then
50+
python data/make_input_embeds_for_quantize.py
51+
```
52+
53+
2. Use the following code to export the RKLLM model.
54+
55+
```bash
56+
python export/export_rkllm.py
57+
```
58+
59+
## 4. C++ Demo
60+
In the `deploy` directory, we provide example code for board-side inference. This code demonstrates the process of "image input to image features," where an input image is processed to output its corresponding image features. These features are then used by the RKLLM model for multimodal content inference.
61+
62+
### 1. Compile and Build
63+
Users can directly compile the example code by running the `deploy/build-linux.sh` or `deploy/build-android.sh` script (replacing the cross-compiler path with the actual path). This will generate an `install/demo_Linux_aarch64` folder in the `deploy` directory, containing the executables `imgenc`, `llm`, `demo`, and the `lib` folder.
64+
65+
```bash
66+
cd deploy
67+
# for linux
68+
./build-linux.sh
69+
# for android
70+
./build-android.sh
71+
# push install dir to device
72+
adb push ./install/demo_Linux_aarch64 /data
73+
# push model file to device
74+
adb push qwen2_vl_2b_vision_rk3588.rknn /data/models
75+
adb push Qwen2-VL-2B-Instruct.rkllm /data/models
76+
# push demo image to device
77+
adb push ../data/demo.jpg /data/demo_Linux_aarch64
78+
```
79+
80+
### 2. Run Demo
81+
Enter the `/data/demo_Linux_aarch64` directory on the board and run the example using the following code
82+
83+
```bash
84+
adb shell
85+
cd /data/demo_Linux_aarch64
86+
# export lib path
87+
export LD_LIBRARY_PATH=./lib
88+
# soft link models dir
89+
ln -s /data/models .
90+
# run imgenc
91+
./imgenc models/qwen2_vl_2b_vision_rk3588.rknn demo.jpg
92+
# run llm(Pure Text Example)
93+
./llm models/Qwen2-VL-2B-Instruct.rkllm 128 512
94+
# run demo(Multimodal Example)
95+
./demo demo.jpg models/qwen2_vl_2b_vision_rk3588.rknn models/Qwen2-VL-2B-Instruct.rkllm 128 512
96+
```
97+
98+
The user can view the relevant runtime logs in the terminal and obtain the `img_vec.bin` file in the current directory, which contains the image features corresponding to the input image.
99+
100+
Multimodal Example
101+
102+
```
103+
user: <image>What is in the image?
104+
robot: The image depicts an astronaut on the moon, enjoying a beer. The background shows the Earth and stars, creating a surreal and futuristic scene.
105+
```
106+
107+
Pure Text Example
108+
```
109+
user: 把这句话翻译成英文: RK3588是新一代高端处理器,具有高算力、低功耗、超强多媒体、丰富数据接口等特点
110+
robot: The RK3588 is a new generation of high-end processors with high computational power, low power consumption, strong multimedia capabilities, and rich data interfaces.
111+
```
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
[
2+
{"image_path": "data/datasets", "image": "1.jpg", "input": "Question: What is correct Python code to generate the content of the image?\nOptions:\nA. for x in range(6):\n print(x)\nelse:\n print(\"Finally finished!\")\n\nB. thisdict = {\n \"brand\": \"Ford\",\n \"model\": \"Mustang\",\n \"year\": 1964\n}\n\nprint(len(thisdict))\nC. x = 1\ny = 2.8\nz = 1j\n\nprint(type(x))\nprint(type(y))\nprint(type(z))\n\nD. fruits = [\"apple\", \"banana\", \"cherry\"]\nfor x in fruits:\n print(x)\nPlease select the correct answer from the options above. \n", "target":"D"},
3+
{"image_path": "data/datasets", "image": "2.jpg", "input": "Question: What is correct Python code to generate the content of the image?\nOptions:\nA. class Person:\n def __init__(self, name, age):\n self.name = name\n self.age = age\n\np1 = Person(\"John\", 36)\n\nprint(p1.name)\nprint(p1.age)\nB. fruits = [\"apple\", \"banana\", \"cherry\"]\nfor x in fruits:\n print(x)\nC. x = min(5, 10, 25)\ny = max(5, 10, 25)\n\nprint(x)\nprint(y)\nD. a = 33\nb = 200\nif b > a:\n print(\"b is greater than a\")\nPlease select the correct answer from the options above. \n", "target":"D"},
4+
{"image_path": "data/datasets", "image": "21.jpg", "input": "Question: Which one is the correct caption of this image?\nOptions:\nA. A man rides a surfboard on a large wave.\nB. a young boy barefoot holding an umbrella touching the horn of a cow\nC. A giraffe standing by a stall in a field.\nD. A stop sign that has been vandalized with graffiti.\nPlease select the correct answer from the options above. \n", "target":"B"},
5+
{"image_path": "data/datasets", "image": "22.jpg", "input": "Question: Which one is the correct caption of this image?\nOptions:\nA. A narrow kitchen filled with appliances and cooking utensils.\nB. A person with glasses and a tie in a room.\nC. Tray of vegetables with cucumber, carrots, broccoli and celery.\nD. A pretty young woman riding a surfboard on a wave in the ocean.\nPlease select the correct answer from the options above. \n", "target":"A"},
6+
{"image_path": "data/datasets", "image": "241.jpg", "input": "Hint: The passage below describes an experiment. Read the passage and then follow the instructions below.\n\nMadelyn applied a thin layer of wax to the underside of her snowboard and rode the board straight down a hill. Then, she removed the wax and rode the snowboard straight down the hill again. She repeated the rides four more times, alternating whether she rode with a thin layer of wax on the board or not. Her friend Tucker timed each ride. Madelyn and Tucker calculated the average time it took to slide straight down the hill on the snowboard with wax compared to the average time on the snowboard without wax.\nFigure: snowboarding down a hill.\nQuestion: Identify the question that Madelyn and Tucker's experiment can best answer.\nOptions:\nA. Does Madelyn's snowboard slide down a hill in less time when it has a thin layer of wax or a thick layer of wax?\nB. Does Madelyn's snowboard slide down a hill in less time when it has a layer of wax or when it does not have a layer of wax?\nPlease select the correct answer from the options above. \n", "target":"B"},
7+
{"image_path": "data/datasets", "image": "252.jpg", "input": "Hint: People can use the engineering-design process to develop solutions to problems. One step in the process is testing if a potential solution meets the requirements of the design.\nThe passage below describes how the engineering-design process was used to test a solution to a problem. Read the passage. Then answer the question below.\n\nLaura and Isabella were making batches of concrete for a construction project. To make the concrete, they mixed together dry cement powder, gravel, and water. Then, they checked if each batch was firm enough using a test called a slump test.\nThey poured some of the fresh concrete into an upside-down metal cone. They left the concrete in the metal cone for 30 seconds. Then, they lifted the cone to see if the concrete stayed in a cone shape or if it collapsed. If the concrete in a batch collapsed, they would know the batch should not be used.\nFigure: preparing a concrete slump test.\nQuestion: Which of the following could Laura and Isabella's test show?\nOptions:\nA. if the concrete from each batch took the same amount of time to dry\nB. if a new batch of concrete was firm enough to use\nPlease select the correct answer from the options above. \n", "target":"B"},
8+
{"image_path": "data/datasets", "image": "362.jpg", "input": "Hint: Native copper has the following properties:\nsolid\nnot made by living things\nfound in nature\nfixed crystal structure\nmade of the metal copper\nQuestion: Is native copper a mineral?\nOptions:\nA. no\nB. yes\nPlease select the correct answer from the options above. \n", "target":"B"},
9+
{"image_path": "data/datasets", "image": "364.jpg", "input": "Hint: Plastic has the following properties:\nsolid\nno fixed crystal structure\nnot a pure substance\nmade in a factory\nQuestion: Is plastic a mineral?\nOptions:\nA. yes\nB. no\nPlease select the correct answer from the options above. \n", "target":"B"},
10+
{"image_path": "data/datasets", "image": "448.jpg", "input": "Hint: Read the text.\nButterflies and moths are easily mistaken for each other, but one distinction between them often appears during their pupal stage. When most butterfly caterpillars reach full size, they attach themselves to a leaf or other object and shed their skin a final time, forming a chrysalis, a hard, shell-like skin, which protects the pupa inside. The chrysalis may be dull and rough or shiny and smooth, usually blending into its surroundings. Most moth caterpillars, by contrast, create a cocoon to protect the pupa, rather than forming a chrysalis. The cocoons usually resemble hard silk pouches, but some moths also incorporate materials like hairs and twigs.\nQuestion: Which term matches the picture?\nOptions:\nA. cocoon\nB. chrysalis\nPlease select the correct answer from the options above. \n", "target":"B"},
11+
{"image_path": "data/datasets", "image": "477.jpg", "input": "Hint: Read the text.\nHeat transfer can occur in different ways. Two common ways are through conduction and convection. Conduction occurs when molecules from one object collide with molecules from another object. Burning your hand by touching a hot car door on a sunny summer day is an example of conduction.\nConvection is another form of heat transfer. When a liquid or gas is heated, the heated matter rises upward, away from the heat source. Hot bubbles rising in a pot of water boiling on a stove is an example of convection.\nQuestion: Which term matches the picture?\nOptions:\nA. conduction\nB. convection\nPlease select the correct answer from the options above. \n", "target":"B"},
12+
{"image_path": "data/datasets", "image": "1231.jpg", "input": "Question: Which image is more brightful?\nOptions:\nA. The first image\nB. The second image\nPlease select the correct answer from the options above. \n", "target":"A"},
13+
{"image_path": "data/datasets", "image": "1232.jpg", "input": "Question: Which image is more brightful?\nOptions:\nA. The first image\nB. The second image\nPlease select the correct answer from the options above. \n", "target":"A"},
14+
{"image_path": "data/datasets", "image": "1085.jpg", "input": "Question: is this place crowded?\nOptions:\nA. yes\nB. no\nPlease select the correct answer from the options above. \n", "target":"A"},
15+
{"image_path": "data/datasets", "image": "1086.jpg", "input": "Question: is this place crowded?\nOptions:\nA. yes\nB. no\nPlease select the correct answer from the options above. \n", "target":"A"},
16+
{"image_path": "data/datasets", "image": "1128.jpg", "input": "Question: In this picture, are the two dolphins the same size?\nOptions:\nA. same\nB. Not the same\nC. Can't judge\nPlease select the correct answer from the options above. \n", "target":"B"},
17+
{"image_path": "data/datasets", "image": "1129.jpg", "input": "Question: In this picture, are the two butterfly wings the same shape?\nOptions:\nA. same\nB. Not the same\nC. Can't judge\nPlease select the correct answer from the options above. \n", "target":"B"},
18+
{"image_path": "data/datasets", "image": "1200.jpg", "input": "Question: What will happen next?\nOptions:\nA. the motorcyle is gonna go forward\nB. the motorcyle is gonna crash\nC. the motorcyle is gonna go backward\nD. both A,B, and C\nPlease select the correct answer from the options above. \n", "target":"B"},
19+
{"image_path": "data/datasets", "image": "1201.jpg", "input": "Question: What will happen next?\nOptions:\nA. this person is gonna stay still\nB. this person is gonna keep walking\nC. this person is gonna fall into the water\nD. both A,B, and C\nPlease select the correct answer from the options above. \n", "target":"C"},
20+
{"image_path": "data/datasets", "image": "1554.jpg", "input": "Question: The object shown in this figure:\nOptions:\nA. Is a colorless, flammable liquid that is commonly used as a solvent and fuel\nB. Has a boiling point of 64.7°C\nC. Can be toxic if ingested or absorbed through the skin\nD. None of these options are correct.\nPlease select the correct answer from the options above. \n", "target":"C"},
21+
{"image_path": "data/datasets", "image": "1555.jpg", "input": "Question: The object shown in this figure:\nOptions:\nA. Is a lustrous, white metal that is highly reflective and ductile\nB. Has the highest electrical and thermal conductivity of all metals\nC. Has a boiling point of 2,162°C\nD. All of these options are correct.\nPlease select the correct answer from the options above. \n", "target":"D"}
22+
]
2.63 KB
Loading

0 commit comments

Comments
 (0)