mirror of
https://github.com/sogou/workflow.git
synced 2026-10-21 22:56:42 +08:00
Merge pull request #504 from dengjunplusplus/windows
update README of windows branch
This commit is contained in:
@@ -1,112 +1,187 @@
|
||||
[English version](README_en.md)
|
||||
[Workflow Introduction](https://github.com/sogou/workflow/blob/master/README.md)
|
||||
|
||||
[Windows下Workflow编译指南](Readme_cn.md)
|
||||
|
||||
[](https://git.sogou-inc.com/wujiaxu/Filter/blob/master/LICENSE)
|
||||
[](https://en.cppreference.com/)
|
||||
[](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9)
|
||||
[](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9)
|
||||
[](https://travis-ci.com/sogou/workflow)
|
||||
|
||||
# Sogou C++ Workflow
|
||||
#### 搜狗公司的后端C++编程标准,是一套企业级的程序引擎。主要功能和特点:
|
||||
* 这是一个基于C++11 ``std::function`` 的异步引擎。用于解决一切关于串行,并行和异步的问题。
|
||||
* 作为网络框架,完全协议无关,并且直接面向应用。
|
||||
* 可以当作一个异步redis客户端,也可以快速搭建一个Http服务器。
|
||||
* 自定义协议非常方便,你可以快速的建立一个自己的RPC系统。
|
||||
* Sogou RPC就是以它为基础开发,作为独立项目开源。该项目支持srpc,brpc和thrift等协议([benchmark](https://github.com/holmes1412/sogou-rpc-benchmark))。
|
||||
* 支持SSL(依赖openssl)。支持TCP, UDP,SCTP等常用传输层协议。可支持SCTP上的SSL。但不支持UDP server。
|
||||
* 原生包含了多种常有互联网协议的实现,并且以统一的方式使用。
|
||||
* 目前支持http,redis,mysql和kafka协议,可以直接访问这些资源或搭建这些协议的server。
|
||||
* 可能是目前市面上唯一C++全功能的mysql异步客户端。
|
||||
* DNS协议开发中,目前使用系统库访问dns。
|
||||
* 包含强大的计算任务调度功能。
|
||||
* 计算任务可以和通信任务一样都可以加入任务流,有各自的调度器实现调度。
|
||||
* 你可以不用网络功能,把它当并行编程工具来使用。
|
||||
* 我们最大的目的就是在计算和通信非常复杂的情况下,最大程度的发挥节点的性能。
|
||||
* 提供了一些常用算法实现,例如并行排序,MapReduce。
|
||||
* 实际上,一切异步过程(如磁盘IO,GPU任务,定时器等)都可以被协同调度。
|
||||
* 如果用户使用Linux系统,磁盘IO任务是是通过Linux底层aio实现,效率极高。
|
||||
* 支持任何DAG结构的任务流。但大多数情况下,用户只需要使用串并联结构。
|
||||
* 自带负载均衡和强大的服务治理等功能。
|
||||
* 可以方便的与其他异步引擎协同使用。
|
||||
* 支持streaming的通讯引擎开发中。
|
||||
* 作为server使用时,支持多进程模式,支持精确的优雅重启。
|
||||
# Build Workflow On Windows
|
||||
|
||||
#### 相关的技术要点:
|
||||
* 目前项目支持Linux,macOS,FreeBSD,Windows等系统。需要安装cmake。
|
||||
* Windows版暂时以独立[branch](https://github.com/sogou/workflow/tree/windows)发布,以iocp为异步通讯基础,对外接口一致。
|
||||
* 该项目使用C/C++编写,使用者需要能比较熟练的使用C++编程。不依赖boost或asio,编译速度极快。
|
||||
* 用到少量的C++11特征,用户需要会使用 ``std::function`` 和 ``std::move()`` 。
|
||||
* 理论支持一切CPU架构,可以在32位或64位arm处理器上编译运行。Big endian CPU未测试。
|
||||
* 项目需要依赖openssl。如果用户对SSL性能要求高,强烈建议使用openssl 1.1以上版本。
|
||||
* 项目无其他依赖,但包含了snappy,lz4等几个压缩库的无改动源代码(kafka协议需要)。
|
||||
We use cmake to build the workflow project. It denpends one OpenSSL. It's essential to install **CMake and OpenSSL**.
|
||||
|
||||
#### 项目的一些设计特点:
|
||||
* 项目基础用法非常简单,极易上手。而我们的一些设计也大幅降低了一般C++项目的使用难度。
|
||||
* 尽量避免让用户派生,一切用户行为都用 ``std::function`` 来包装,例如:
|
||||
* 任何任务结束后的callback。
|
||||
* 计算任务里的算法。
|
||||
* 一个server,也对应一个 ``std::function`` 。
|
||||
* 尽量避免复杂的内存管理,一切任务和框架由工厂类产生,并自动内存回收。
|
||||
* 任何任务在callback之后被自动delete。
|
||||
* 任务中的数据(例如一个网络回复包,一个算法的结果),如果用户要保留,需要用 ``std::move()`` 把它移走。
|
||||
* 内存回收是使用一套严密又符合自然逻辑的机制,我们不使用任何share_ptr。
|
||||
* 尽量避免复杂的参数配置。
|
||||
* 我们可配置的参数非常多,但您可以在对参数完全无感的情况下使用我们的系统。
|
||||
* 如果您对程序行为,资源配比有特定需求,一定能找到对应配置项,以便发挥程序最高性能。
|
||||
* 项目采用全异步的设计,并且对用户不透明,用户需要知道自己在写异步程序。
|
||||
* 我们通过精心的设计的用户尽量简单的使用异步,得益于 ``std::function`` 带来的便利性,以及自动内存回收机制。
|
||||
* 没有采用用户态线程的方式,一方面是性能考虑,另一方面原因是我们有计算任务(线程任务)调度的概念。
|
||||
* 在我们的设计里,计算是一种异步任务,和通信并没有什么区别。
|
||||
* 计算任务由独立的线程组依照特定算法进行调度,并不一定实时被执行。
|
||||
* 因为有这种计算任务的存在,用户态线程似乎就没什么意义了,用户必然需要了解异步。
|
||||
* 因为全异步的原因,我们几乎所有的核心调用都是短操作,不会阻塞。
|
||||
* 我们也不鼓励用户在callback里阻塞程序,或做复杂的计算。但如果业务逻辑简单,这么做也无妨。
|
||||
* 项目使用方法的简单总结:
|
||||
* 用户如同搭建串并联电路一样搭建程序。电路可以一开始就生成好,也可以运行中动态产生。
|
||||
* 我们提供各种电子元件供用户使用,一次http请求,一次GPU矩阵乘法,一次并行排序,都可以理解成为一个元件。
|
||||
* 每个电子元件都有标准的输入输出。但每个电子元件内部,可能又是一个复杂电路,但用户并不感知。
|
||||
* 比如一个http请求,可能要经历DNS,redirect,retry等多次异步过程,但对用户来讲,只是一个元件。
|
||||
* 用户可以方便的定义自己的元件。比如一个算法,一次通信。
|
||||
* 无状态的协议实现起来极其简单。如果协议有登录,选库等过程,则会复杂一些,可参考redis实现。
|
||||
* 通过强大的Upstream系统,能实现非常复杂的服务治理,例如通信节点选择,负载均衡,熔断与恢复,主备等。
|
||||
* 总之这是一个企业级的,设计优雅的异步编程框架,几乎可以覆盖一切高性能的后端服务需求。
|
||||
## Install CMake
|
||||
|
||||
#### 使用教程:
|
||||
* Client基础
|
||||
* [创建第一个任务:wget](docs/tutorial-01-wget.md)
|
||||
* [实现一次redis写入与读出:redis_cli](docs/tutorial-02-redis_cli.md)
|
||||
* [任务序列的更多功能:wget_to_redis](docs/tutorial-03-wget_to_redis.md)
|
||||
* Server基础
|
||||
* [第一个server:http_echo_server](docs/tutorial-04-http_echo_server.md)
|
||||
* [异步server的示例:http_proxy](docs/tutorial-05-http_proxy.md)
|
||||
* 并行任务与工作流
|
||||
* [一个简单的并行抓取:parallel_wget](docs/tutorial-06-parallel_wget.md)
|
||||
* 几个重要的话题
|
||||
* [关于错误处理](docs/about-error.md)
|
||||
* [关于超时](docs/about-timeout.md)
|
||||
* [关于DNS](docs/about-dns.md)
|
||||
* [关于程序退出](docs/about-exit.md)
|
||||
* 计算任务
|
||||
* [使用内置算法工厂:sort_task](docs/tutorial-07-sort_task.md)
|
||||
* [自定义计算任务:matrix_multiply](docs/tutorial-08-matrix_multiply.md)
|
||||
* 文件异步IO任务
|
||||
* [异步IO的http server:http_file_server](docs/tutorial-09-http_file_server.md)
|
||||
* 用户定义协议基础
|
||||
* [简单的用户自定义协议client/server](docs/tutorial-10-user_defined_protocol.md)
|
||||
* 定时与计数任务
|
||||
* [关于定时器](docs/about-timer.md)
|
||||
* [关于计数器](docs/about-counter.md)
|
||||
* 服务治理
|
||||
* [关于服务治理](docs/about-service-management.md)
|
||||
* [Upstream更多文档](docs/about-upstream.md)
|
||||
* 连接上下文的使用
|
||||
* [关于连接上下文](docs/about-connection-context.md)
|
||||
* 内置协议用法
|
||||
* [异步MySQL客户端:mysql_cli](docs/tutorial-12-mysql_cli.md)
|
||||
Visit [CMake Homepage](https://cmake.org/download/), and download the msi installer to install cmake.
|
||||
|
||||
## Authors
|
||||
## Install OpenSSL
|
||||
|
||||
### Ways to Install OpenSSL Library On Windos
|
||||
|
||||
* Download the source code to compile
|
||||
* Donnload the bianary installer of OpenSSL
|
||||
* Install by Chocolatey
|
||||
* Install by VCPKG
|
||||
|
||||
### Compile OpenSSL by Source Code
|
||||
|
||||
Visit [OpenSSL Homepage](https://www.openssl.org/) or [Github repository](https://github.com/openssl/openssl) to download the source code.
|
||||
Compile OpenSSL by its document.
|
||||
|
||||
### Install OpenSSL by binary installer
|
||||
|
||||
Google *OpenSSL binary installer* and download the installer to install. **Do not use light version**, because it doesn't contains *include and lib dictionary*.
|
||||
|
||||
[Here](https://slproweb.com/products/Win32OpenSSL.html) is a binary installer download website.
|
||||
|
||||
|
||||
### Install OpenSSL by Chocolatey
|
||||
|
||||
[Chocolatey](https://community.chocolatey.org/) is a powerful package manager on windows. It's like apt-get or yum on some Linux distribution.
|
||||
|
||||
Use the command to install OpenSSL by Chocolatey.
|
||||
|
||||
```powershell
|
||||
choco install openssl
|
||||
```
|
||||
|
||||
### Install OpenSSL by VCPKG
|
||||
|
||||
[VCPKG](https://docs.microsoft.com/zh-cn/cpp/build/vcpkg?view=vs-2019) is a powerful CPP package manager across platforms.
|
||||
It simplify the installation of some libraries.
|
||||
|
||||
Use the command to install OpenSSL by vcpkg.
|
||||
|
||||
```powershell
|
||||
vcpkg install openssl
|
||||
```
|
||||
|
||||
## Compile Workflow
|
||||
### Compile OpenSSL by source code
|
||||
|
||||
Specify the **OPENSSL_ROOT_DIR** to generate Visual Studio Project, Such as
|
||||
|
||||
```powershell
|
||||
cmake -B [build directory] -S . -DOPENSSL_ROOT_DIR=[openssl directory]
|
||||
```
|
||||
|
||||
### Install OpenSSL by Installer or Chocolatey
|
||||
|
||||
Use Command to generate Visual Studio Project
|
||||
|
||||
```powershell
|
||||
cmake -B [build directory] -S .
|
||||
```
|
||||
|
||||
### Install OpenSSL by vcpkg
|
||||
Specify **CMAKE_TOOLCHAIN_FILE** is essential to generate Visual Studio Project, Suche as
|
||||
|
||||
```powershell
|
||||
cmake -B [build directory] -S . -DCMAKE_TOOLCHAIN_FILE=[vcpkg.cmake directory]
|
||||
# If the above command fails, try to specify VCPKG_TARGET_TRIPLET, use x86-windows or x64-windows
|
||||
cmake -B [build directory] -S . -DVCPKG_TARGET_TRIPLET=x86-windows -DCMAKE_TOOLCHAIN_FILE=[vcpkg.cmake directory]
|
||||
```
|
||||
|
||||
### Others
|
||||
**[openssl directory]**: openssl directory of source code
|
||||
|
||||
**[vcpkg.cmake directory]** : [vcpkg-root]\scripts\buildsystems\vcpkg.cmake
|
||||
|
||||
|
||||
The VS Project(workflo.sln) will be generated now.
|
||||
|
||||
Use VS to open workflow.sln or use cmake command to compile workflow, Such as
|
||||
|
||||
```powershell
|
||||
# Compile Debug Config
|
||||
cmake --build [build directory] --config Debug
|
||||
# Compile Release Config
|
||||
cmake --build [build directory] --config Release
|
||||
```
|
||||
|
||||
### Compile Tutorial of Workflow
|
||||
```powershell
|
||||
|
||||
# Generate VS Project of tutorial
|
||||
cmake -B build_tutorial tutorial
|
||||
|
||||
# Compile Debug Config
|
||||
cmake --build build_tutorial --config Debug
|
||||
# Compile Release Config
|
||||
cmake --build build_tutorial --config Release
|
||||
|
||||
# compile result will be generated in tutorial\Debug or tutorial\Release directory
|
||||
```
|
||||
|
||||
# Install Workflow by VCPKG
|
||||
|
||||
Workflow is already supported by VCPKG. If you don't care the source code of workflow, It's more easier to use vcpkg to install it.
|
||||
|
||||
[VCPKG Guide](https://docs.microsoft.com/zh-cn/cpp/build/vcpkg?view=msvc-160)
|
||||
|
||||
Install current version of workflow:`vcpkg install workflow`
|
||||
|
||||
Install HEAD version of workflow:`vcpkg install workflow --head`
|
||||
|
||||
Some examples:
|
||||
|
||||
## On Windows
|
||||
|
||||
```powershell
|
||||
md D:\tmp
|
||||
cd D:\tmp
|
||||
git clone https://github.com/microsoft/vcpkg.git
|
||||
cd vcpkg
|
||||
.\bootstrap-vcpkg.bat
|
||||
# x86-windows is default, it means .\vcpkg.exe install workflow:x86-windows
|
||||
.\vcpkg.exe install workflow
|
||||
# Specify x64-windows to install X64 version
|
||||
.\vcpkg.exe install workflow:x64-windows
|
||||
|
||||
cd D:\tmp
|
||||
git clone https://github.com/dengjunplusplus/workflow-vcpkg-tutorial
|
||||
|
||||
cd D:\tmp\workflow-vcpkg-tutorial\workflow
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=D:/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -B build
|
||||
# If the above command fails, try to specify VCPKG_TARGET_TRIPLET, use x86-windows or x64-windows
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=D:/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x86-windows -B build
|
||||
cmake --build build --config Debug
|
||||
cmake --build build --config Release
|
||||
|
||||
```
|
||||
|
||||
## On Linux & Mac
|
||||
|
||||
```bash
|
||||
cd /tmp
|
||||
rm -rf vcpkg
|
||||
rm -rf workflow-vcpkg-tutorial
|
||||
git clone https://github.com/microsoft/vcpkg.git
|
||||
cd vcpkg
|
||||
./bootstrap-vcpkg.sh
|
||||
./vcpkg install workflow
|
||||
|
||||
cd ..
|
||||
git clone https://github.com/dengjunplusplus/workflow-vcpkg-tutorial.git
|
||||
cd workflow-vcpkg-tutorial/workflow
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -B build
|
||||
# If the above command fails, try to specify VCPKG_TARGET_TRIPLET, x64-linux or x86-linux or x64-osx
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-linux -B build
|
||||
cmake --build build --config Debug
|
||||
cmake --build build --config Release
|
||||
|
||||
```
|
||||
|
||||
# Contact Us
|
||||
|
||||
If you have some problems about use Workflow or compile it. Welcome to commit Issue or Email us.
|
||||
|
||||
## Problems about Use Workflow
|
||||
|
||||
* **Xie Han** - *[xiehan@sogou-inc.com](mailto:xiehan@sogou-inc.com)*
|
||||
* **Wu Jiaxu** - *[wujiaxu@sogou-inc.com](mailto:wujiaxu@sogou-inc.com)*
|
||||
* **Li Yingxin** - *[liyingxin@sogou-inc.com](mailto:liyingxin@sogou-inc.com)*
|
||||
|
||||
## Problems about Compile Workflow On Windows Or VCPKG
|
||||
|
||||
* **Deng Jun** - *[dengjun@sogou-inc.com](mailto:dengjun@sogou-inc.com)*
|
||||
+179
@@ -0,0 +1,179 @@
|
||||
[Workflow 中文介绍](https://github.com/sogou/workflow/blob/master/README_cn.md)
|
||||
|
||||
[Windows Build Guide in English](Readme.md)
|
||||
|
||||
[](https://git.sogou-inc.com/wujiaxu/Filter/blob/master/LICENSE)
|
||||
[](https://en.cppreference.com/)
|
||||
[](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9)
|
||||
[](https://travis-ci.com/sogou/workflow)
|
||||
|
||||
# Windows下编译
|
||||
|
||||
Workflow采用CMake作为构建系统,依赖OpenSSL,编译workflow前需要先安装**CMake和OpenSSL**
|
||||
|
||||
## 安装CMake
|
||||
通过[CMake官网](https://cmake.org/download/)下载cmake对应的安装包安装即可
|
||||
|
||||
## 安装OpenSSL
|
||||
|
||||
### Windows下安装OpenSSL库包括以下几种方式
|
||||
|
||||
* 下载源码自行编译
|
||||
* 下载二进制文件或安装包
|
||||
* 通过Chocolatey安装
|
||||
* 通过vcpkg安装
|
||||
|
||||
### 下载源码自行编译
|
||||
|
||||
可以去官网或者GitHub仓库进行代码下载,编译过程参见OpenSSL源码中的编译文档。
|
||||
|
||||
* [openssl官网](https://www.openssl.org/)
|
||||
* [github仓库](https://github.com/openssl/openssl)
|
||||
|
||||
### 下载安装包安装
|
||||
|
||||
请自行通过搜索引擎进行搜索,将文件解压到特定目录,或者下载并运行安装包。
|
||||
安装包下载时,请**不要安装light版本**,因为light版本中不包含include和lib目录,无法正常编译
|
||||
* [安装包下载](https://slproweb.com/products/Win32OpenSSL.html)
|
||||
|
||||
|
||||
### 通过Chocolatey安装
|
||||
|
||||
Chocolatey是一个 Windows下的包管理器,类似于 Linux 下的 apt-get或 yum,可以方便的进行软件包的安装,Chocolatey的安装参考[Chocolatey官网](https://www.chocolatey.org/)
|
||||
安装完Chocolatey后,执行下面的命令,即可安装OpenSSL
|
||||
```powershell
|
||||
choco install openssl
|
||||
```
|
||||
|
||||
### 通过vcpkg安装
|
||||
|
||||
vcpkg是一个Windows下的C++包管理工具,它极大地简化了第三方库的配置与安装,vcpkg的安装请参考微软官方文档[VCPKG官方文档](https://docs.microsoft.com/zh-cn/cpp/build/vcpkg?view=vs-2019)
|
||||
安装完vcpkg后,执行下面的命令,即可安装OpenSSL
|
||||
```powershell
|
||||
vcpkg install openssl
|
||||
```
|
||||
|
||||
## 编译workflow库
|
||||
### 通过源码编译OpenSSL,生成VS工程
|
||||
通过源码编译或者下载二进制进行openssl安装的,在编译过程中可以通过指定OPENSSL_ROOT_DIR的方式进行编译,在workflow根目录下执行下面的命令
|
||||
|
||||
```powershell
|
||||
cmake -B [build directory] -S . -DOPENSSL_ROOT_DIR=[openssl directory]
|
||||
```
|
||||
|
||||
### 通过安装包或Chocolatey安装OpenSSL,生成VS工程
|
||||
通过安装包安装或者通过Chocolatey安装openssl的,无需指定openssl的路径,在workflow根目录下执行下面的命令
|
||||
|
||||
```powershell
|
||||
cmake -B [build directory] -S .
|
||||
```
|
||||
|
||||
### 通过vcpkg安装OpenSSL,生成VS工程
|
||||
通过vcpkg安装openssl的,需要指定CMAKE_TOOLCHAIN_FILE的路径
|
||||
|
||||
```powershell
|
||||
cmake -B [build directory] -S . -DCMAKE_TOOLCHAIN_FILE=[vcpkg.cmake directory]
|
||||
# 如果上面执行失败,可以尝试指定DVCPKG_TARGET_TRIPLET。 x86-windows或x64-windows
|
||||
cmake -B [build directory] -S . -DVCPKG_TARGET_TRIPLET=x86-windows -DCMAKE_TOOLCHAIN_FILE=[vcpkg.cmake directory]
|
||||
```
|
||||
|
||||
### 其他
|
||||
**[openssl directory]**: openssl目录,请自行替换
|
||||
|
||||
**[build directory]** : 构建目录,一般使用build目录
|
||||
|
||||
**[vcpkg.cmake directory]** : 一般为vcpkg根目录\scripts\buildsystems\vcpkg.cmake
|
||||
|
||||
执行完成后,在[build directory]下会生成workflow.sln,可以用VS打开进行编译,或执行cmake命令进行编译
|
||||
|
||||
```powershell
|
||||
# 编译Debug版本
|
||||
cmake --build [build directory] --config Debug
|
||||
# 编译Release版本
|
||||
cmake --build [build directory] --config Release
|
||||
```
|
||||
|
||||
### 编译tutorial
|
||||
```powershell
|
||||
|
||||
# 生成tutorial工程
|
||||
cmake -B build_tutorial tutorial
|
||||
|
||||
# 编译Debug版本
|
||||
cmake --build build_tutorial --config Debug
|
||||
# 编译Release版本
|
||||
cmake --build build_tutorial --config Release
|
||||
|
||||
# tutorial\Debug或者tutorial\Release目录下即为tutorial编译结果
|
||||
```
|
||||
|
||||
# VCPKG安装workflow
|
||||
|
||||
Workflow已经支持VCPKG安装啦!**如果只是使用workflow,而不关注源码,可以使用VCPKG安装**。
|
||||
|
||||
VCPKG如何安装及使用详见[官方文档](https://docs.microsoft.com/zh-cn/cpp/build/vcpkg?view=msvc-160)
|
||||
|
||||
安装workflow当前版本:`vcpkg install workflow`
|
||||
|
||||
安装workflow最新HEAD提交版本:`vcpkg install workflow --head`
|
||||
|
||||
下面是各平台的安装示例:
|
||||
|
||||
## On Windows
|
||||
|
||||
```powershell
|
||||
md D:\tmp
|
||||
cd D:\tmp
|
||||
git clone https://github.com/microsoft/vcpkg.git
|
||||
cd vcpkg
|
||||
.\bootstrap-vcpkg.bat
|
||||
# 默认安装x86-windows,与.\vcpkg.exe install workflow:x86-windows相同
|
||||
.\vcpkg.exe install workflow
|
||||
# 也可以指定安装x64
|
||||
.\vcpkg.exe install workflow:x64-windows
|
||||
|
||||
cd D:\tmp
|
||||
git clone https://github.com/dengjunplusplus/workflow-vcpkg-tutorial
|
||||
|
||||
cd D:\tmp\workflow-vcpkg-tutorial\workflow
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=D:/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -B build
|
||||
# 如果上面执行失败,可以尝试指定VCPKG_TARGET_TRIPLET。 x86-windows或x64-windows
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=D:/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x86-windows -B build
|
||||
cmake --build build --config Debug
|
||||
cmake --build build --config Release
|
||||
|
||||
```
|
||||
|
||||
## On Linux & Mac
|
||||
|
||||
```bash
|
||||
cd /tmp
|
||||
rm -rf vcpkg
|
||||
rm -rf workflow-vcpkg-tutorial
|
||||
git clone https://github.com/microsoft/vcpkg.git
|
||||
cd vcpkg
|
||||
./bootstrap-vcpkg.sh
|
||||
./vcpkg install workflow
|
||||
|
||||
cd ..
|
||||
git clone https://github.com/dengjunplusplus/workflow-vcpkg-tutorial.git
|
||||
cd workflow-vcpkg-tutorial/workflow
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -B build
|
||||
# 如果上面执行失败,可以尝试指定DVCPKG_TARGET_TRIPLET。 x64-linux 或x86-linux或x64-osx
|
||||
cmake -DCMAKE_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-linux -B build
|
||||
cmake --build build --config Debug
|
||||
cmake --build build --config Release
|
||||
|
||||
```
|
||||
|
||||
# 联系方式
|
||||
如果编译和使用过程有什么问题,欢迎提Issue,或者通过邮件和QQ群(618773193)联系我们
|
||||
|
||||
## Workflow使用相关问题
|
||||
|
||||
* **Xie Han** - *[xiehan@sogou-inc.com](mailto:xiehan@sogou-inc.com)*
|
||||
* **Li Yingxin** - *[liyingxin@sogou-inc.com](mailto:liyingxin@sogou-inc.com)*
|
||||
|
||||
## Windows分支相关问题或VCPKG使用问题
|
||||
|
||||
* **Deng Jun** - *[dengjun@sogou-inc.com](mailto:dengjun@sogou-inc.com)*
|
||||
-112
@@ -1,112 +0,0 @@
|
||||
[中文版](README.md)
|
||||
|
||||
[](https://git.sogou-inc.com/wujiaxu/Filter/blob/master/LICENSE)
|
||||
[](https://en.cppreference.com/)
|
||||
[](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9)
|
||||
|
||||
# Sogou C++ Workflow
|
||||
#### As the backend C++ programming standard in Sogou, Workflow is an industrial-grade programming engine.
|
||||
#### Main functions and features:
|
||||
* An **asynchronous engine** based on **C++11** ``std::function`` which aims to solve all the **serial, parallel and asynchronous** problems.
|
||||
* As a network framework, it is completely **protocol-agnostic** and directly facing applications.
|
||||
* It can either be used as a Redis **client** or an Http **server**.
|
||||
* Convinient to **customize protocols**, so you can quickly build your own RPC systems.
|
||||
* Sogou RPC is developed based on Sogou Workflow and is open source as an independent project. The project supports srpc, brpc and thrift protocols ([benchmark](https://github.com/holmes1412/sogou-rpc-benchmark)).
|
||||
* Support **SSL** (depends on openssl). Support **TCP, UDP, SCTP** and other common transport layer protocols. Support SSL on **SCTP**. Not support UDP server.
|
||||
* A variety of **common Internet protocol** implementations are natively integrated and used in a unified way.
|
||||
* Currently support **http, redis, mysql** and **kafka** protocols. You can directly access these resources or build **servers** for these protocols.
|
||||
* Highly likely the only C++ full-featured **mysql asynchronous client** on the market.
|
||||
* **DNS** protocol is being developing and currently we use the system library to access DNS.
|
||||
* Powerful feature for **scheduling computing tasks**
|
||||
* Computing task, as well as communication task, can be added into the task flow and they’re scheduled separately by their corresponding scheduler.
|
||||
* You can use it as a parallel programming engine **without** the network features.
|
||||
* Our biggest goal is to **maximize the performance** of every node when the calculation and communication environment is very complex.
|
||||
* Some **common algorithm** implementations are provided, such as parallel sorting and MapReduce.
|
||||
* In fact, all asynchronous processes (such as disk IO, GPU tasks, timers, etc.) **can be scheduled in coordination**.
|
||||
* On the Linux system, the disk IO task is realized through the Linux underlying aio, which is extremely efficient.
|
||||
* Support any task flow with **DAG** structure. However, in most cases, users only need **series-parallel** structure.
|
||||
* Built-in **load balancing** and powerful **service governance** features.
|
||||
* Easily used in conjunction with other asynchronous engines.
|
||||
* **Streaming** communication engine is being developed.
|
||||
* When working as a server, it supports **multi-processes** mode and precisely supports **graceful restart**.
|
||||
|
||||
#### Building
|
||||
* Support **Linux, macOS, FreeBSD, Windows** and other systems so far. Installing **cmake** is necessary.
|
||||
* Windows version is temporarily released as an independent branch, which uses **iocp** as the basis for asynchronous communication and mean while, keeping the **same external interface**.
|
||||
* As written in C/C++, it requires the users being able to proficiently use C++ programing. It **does not** rely on boost or asio, therefor the compiling speed is extremely fast.
|
||||
* It contains a small numbers of C++11 features, so users need to being able to use ``std::function`` and ``std::move``.
|
||||
* Theoretically support all CPU architectures and can be compiled and run on **32-bit** or **64-bit arm processors**. Big endian CPU is not tested.
|
||||
* **openssl** is required. If users expect high performance of SSL, openssl 1.1 or higher is strongly recommended.
|
||||
* **No other dependencies**. Several compression libraries such as snappy and lz4 is contained by their unmodified source (required by the Kafka protocol).
|
||||
|
||||
#### Some features of design
|
||||
* The basic usage is very simple and handy. Some features are designed to greatly reduce the difficulty of programming with general C++ projects.
|
||||
* To **avoid** users to **derive** as much as possible, all user behaviors are wrapped with std::function, for example:
|
||||
* the **callback** after every task ends
|
||||
* the **algorithms** in computing tasks
|
||||
* one server corresponds to one ``std::function``
|
||||
* Trying to avoid complicated memory management, all tasks and frameworks are generated by **factory** classes, and their memory is **recycled automatically**. Which means,
|
||||
* Every task is **automatically deleted after its callback**.
|
||||
* If the users want to keep any data in the task (such as a network reply packet or the result of an algorithm), they need to use ``std::move`` to move it.
|
||||
* We treat memory recycle as a strict and naturally logical mechanism, so we **don’t** use share_ptr.
|
||||
* Avoid using complicated parameter configuration.
|
||||
* Actually we have a lot of **configurable parameters**, though you can use our system **without** feeling the parameters exist.
|
||||
* If you have specific requirements for program behavior and resource ratio, you can definitely find the corresponding parameter configuration items in order to maximize the performance of you program.
|
||||
* The project adopts a fully asynchronous design and is not transparent to users, which means users need to know that they are writing asynchronous programs.
|
||||
* Thanks to the convenience brought by ``std::function`` and the automatic memory recycling mechanism, we have delicately designed **the simply possible usage of asynchrony** for users.
|
||||
* **No** user-mode threads concepts. On the one hand, performance is considered. On the other hand, we have the concept of computing tasks (threaded tasks) scheduling.
|
||||
* In our design, **computing** is one kind of **asynchronous task**, which has no differences from communication.
|
||||
* Computing tasks are scheduled by **independent thread groups** according to specific algorithms, please note that they **may not** be executed **immediately**.
|
||||
* As we have such computing tasks, user-mode threads become meaningless, and therefore users must understand asynchrony.
|
||||
* Because of the full asynchrony, almost all core calls are **short** and **non-blocking** operations.
|
||||
* That’s why we **don’t** recommend users to **block** their programs in callback or do some complex calculations. However, it acceptable if the logic is quite simple.
|
||||
* Brief summary of the usage:
|
||||
* The user builds the program just like building a **series-parallel** circuit. The circuit can be generated **at the beginning** or **dynamically during the program running**.
|
||||
* **We provide various electronic components** for users. For instance, one http request, one GPU matrix multiplication, and one parallel sorting can all be understood as a electronic component.
|
||||
* Every electronic component has its **standard input and output**. At the meantime, every electronic component can be a **complicated circuit**, which has no necessary to be perceived by the users.
|
||||
* For example, an http request may go through **multiple asynchronous processes** such as DNS, redirect, and retry, but the entire processes is just a **component** in the perspective of the users.
|
||||
* Users can easily **define their own** components, including algorithms and some kind of communication.
|
||||
* To implement **stateless protocols** is extremely simple. It may be a little bit complicated when the protocol includes login, library selection, etc., at this time, you can refer to the redis implementation.
|
||||
* Through the powerful Upstream system, complex **service governance** can be realized, such as communication node selection, load balancing, circuit breaker and recovery, main and backup, etc.
|
||||
* **In conclusion, this is an enterprise-level, elegantly designed asynchronous framework which can cover almost all high-performance back-end service requirements.**
|
||||
|
||||
#### Tutorials:
|
||||
* Client
|
||||
* [Create your first task:wget](docs/tutorial-01-wget.md)
|
||||
* [Implement redis set and get:redis_cli](docs/tutorial-02-redis_cli.md)
|
||||
* [More features about series:wget_to_redis](docs/tutorial-03-wget_to_redis.md)
|
||||
* Server
|
||||
* [first server:http_echo_server](docs/tutorial-04-http_echo_server.md)
|
||||
* [asynchronous server:http_proxy](docs/tutorial-05-http_proxy.md)
|
||||
* Paralle task and Series
|
||||
* [A simple parallel wget:parallel_wget](docs/tutorial-06-parallel_wget.md)
|
||||
* Important topic
|
||||
* [About error](docs/about-error.md)
|
||||
* [About timeout](docs/about-timeout.md)
|
||||
* [About DNS](docs/about-dns.md)
|
||||
* [About exit](docs/about-exit.md)
|
||||
* Computing tasks
|
||||
* [Using the build-in algorithm factory:sort_task](docs/tutorial-07-sort_task.md)
|
||||
* [User-define computing task:matrix_multiply](docs/tutorial-08-matrix_multiply.md)
|
||||
* File asynchronous IO tasks
|
||||
* [Http server with file IO:http_file_server](docs/tutorial-09-http_file_server.md)
|
||||
* User-defined protocol basic usage
|
||||
* [A simple user-defined portocol client/server](docs/tutorial-10-user_defined_protocol.md)
|
||||
* Timing tasks and counting tasks
|
||||
* [About timer](docs/about-timer.md)
|
||||
* [About counter](docs/about-counter.md)
|
||||
* Service governance
|
||||
* [About service governance](docs/about-service-management.md)
|
||||
* [More documents about upstream](docs/about-upstream.md)
|
||||
* Connection context
|
||||
* [About connection context](docs/about-connection-context.md)
|
||||
* Built-in protocols
|
||||
* [Asynchronous MySQL client:mysql_cli](docs/tutorial-12-mysql_cli.md)
|
||||
|
||||
## Authors
|
||||
|
||||
* **Xie Han** - *[xiehan@sogou-inc.com](mailto:xiehan@sogou-inc.com)*
|
||||
* **Wu Jiaxu** - *[wujiaxu@sogou-inc.com](mailto:wujiaxu@sogou-inc.com)*
|
||||
* **Li Yingxin** - *[liyingxin@sogou-inc.com](mailto:liyingxin@sogou-inc.com)*
|
||||
|
||||
|
||||
Reference in New Issue
Block a user