Merge pull request #504 from dengjunplusplus/windows

update README of windows branch
This commit is contained in:
xiehan
2021-07-31 22:41:30 +08:00
committed by GitHub
3 changed files with 352 additions and 210 deletions
+173 -98
View File
@@ -1,112 +1,187 @@
[English version](README_en.md)
[Workflow Introduction](https://github.com/sogou/workflow/blob/master/README.md)
[Windows下Workflow编译指南](Readme_cn.md)
[![license MIT](https://img.shields.io/badge/License-Apache-yellow.svg)](https://git.sogou-inc.com/wujiaxu/Filter/blob/master/LICENSE)
[![C++](https://img.shields.io/badge/language-c++-red.svg)](https://en.cppreference.com/)
[![platform](https://img.shields.io/badge/platform-linux%20%7C%20macos-lightgrey.svg)](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9)
[![platform](https://img.shields.io/badge/platform-linux%20%7C%20macos%20%7C%20windows-lightgrey.svg)](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9)
[![Build Status](https://travis-ci.com/sogou/workflow.svg?branch=windows)](https://travis-ci.com/sogou/workflow)
# Sogou C++ Workflow
#### 搜狗公司的后端C++编程标准,是一套企业级的程序引擎。主要功能和特点:
* 这是一个基于C++11 ``std::function`` 的异步引擎。用于解决一切关于串行,并行和异步的问题。
* 作为网络框架,完全协议无关,并且直接面向应用。
* 可以当作一个异步redis客户端,也可以快速搭建一个Http服务器。
* 自定义协议非常方便,你可以快速的建立一个自己的RPC系统。
* Sogou RPC就是以它为基础开发,作为独立项目开源。该项目支持srpc,brpc和thrift等协议([benchmark](https://github.com/holmes1412/sogou-rpc-benchmark))。
* 支持SSL(依赖openssl)。支持TCP, UDP,SCTP等常用传输层协议。可支持SCTP上的SSL。但不支持UDP server。
* 原生包含了多种常有互联网协议的实现,并且以统一的方式使用。
* 目前支持http,redis,mysql和kafka协议,可以直接访问这些资源或搭建这些协议的server。
* 可能是目前市面上唯一C++全功能的mysql异步客户端。
* DNS协议开发中,目前使用系统库访问dns。
* 包含强大的计算任务调度功能。
* 计算任务可以和通信任务一样都可以加入任务流,有各自的调度器实现调度。
* 你可以不用网络功能,把它当并行编程工具来使用。
* 我们最大的目的就是在计算和通信非常复杂的情况下,最大程度的发挥节点的性能。
* 提供了一些常用算法实现,例如并行排序,MapReduce。
* 实际上,一切异步过程(如磁盘IO,GPU任务,定时器等)都可以被协同调度。
* 如果用户使用Linux系统,磁盘IO任务是是通过Linux底层aio实现,效率极高。
* 支持任何DAG结构的任务流。但大多数情况下,用户只需要使用串并联结构。
* 自带负载均衡和强大的服务治理等功能。
* 可以方便的与其他异步引擎协同使用。
* 支持streaming的通讯引擎开发中。
* 作为server使用时,支持多进程模式,支持精确的优雅重启。
# Build Workflow On Windows
#### 相关的技术要点:
* 目前项目支持Linux,macOS,FreeBSD,Windows等系统。需要安装cmake。
* Windows版暂时以独立[branch](https://github.com/sogou/workflow/tree/windows)发布,以iocp为异步通讯基础,对外接口一致。
* 该项目使用C/C++编写,使用者需要能比较熟练的使用C++编程。不依赖boost或asio,编译速度极快。
* 用到少量的C++11特征,用户需要会使用 ``std::function`` 和 ``std::move()`` 。
* 理论支持一切CPU架构,可以在32位或64位arm处理器上编译运行。Big endian CPU未测试。
* 项目需要依赖openssl。如果用户对SSL性能要求高,强烈建议使用openssl 1.1以上版本。
* 项目无其他依赖,但包含了snappy,lz4等几个压缩库的无改动源代码(kafka协议需要)。
We use cmake to build the workflow project. It denpends one OpenSSL. It's essential to install **CMake and OpenSSL**.
#### 项目的一些设计特点:
* 项目基础用法非常简单,极易上手。而我们的一些设计也大幅降低了一般C++项目的使用难度。
* 尽量避免让用户派生,一切用户行为都用 ``std::function`` 来包装,例如:
* 任何任务结束后的callback。
* 计算任务里的算法。
* 一个server,也对应一个 ``std::function`` 。
* 尽量避免复杂的内存管理,一切任务和框架由工厂类产生,并自动内存回收。
* 任何任务在callback之后被自动delete。
* 任务中的数据(例如一个网络回复包,一个算法的结果),如果用户要保留,需要用 ``std::move()`` 把它移走。
* 内存回收是使用一套严密又符合自然逻辑的机制,我们不使用任何share_ptr。
* 尽量避免复杂的参数配置。
* 我们可配置的参数非常多,但您可以在对参数完全无感的情况下使用我们的系统。
* 如果您对程序行为,资源配比有特定需求,一定能找到对应配置项,以便发挥程序最高性能。
* 项目采用全异步的设计,并且对用户不透明,用户需要知道自己在写异步程序。
* 我们通过精心的设计的用户尽量简单的使用异步,得益于 ``std::function`` 带来的便利性,以及自动内存回收机制。
* 没有采用用户态线程的方式,一方面是性能考虑,另一方面原因是我们有计算任务(线程任务)调度的概念。
* 在我们的设计里,计算是一种异步任务,和通信并没有什么区别。
* 计算任务由独立的线程组依照特定算法进行调度,并不一定实时被执行。
* 因为有这种计算任务的存在,用户态线程似乎就没什么意义了,用户必然需要了解异步。
* 因为全异步的原因,我们几乎所有的核心调用都是短操作,不会阻塞。
* 我们也不鼓励用户在callback里阻塞程序,或做复杂的计算。但如果业务逻辑简单,这么做也无妨。
* 项目使用方法的简单总结:
* 用户如同搭建串并联电路一样搭建程序。电路可以一开始就生成好,也可以运行中动态产生。
* 我们提供各种电子元件供用户使用,一次http请求,一次GPU矩阵乘法,一次并行排序,都可以理解成为一个元件。
* 每个电子元件都有标准的输入输出。但每个电子元件内部,可能又是一个复杂电路,但用户并不感知。
* 比如一个http请求,可能要经历DNS,redirect,retry等多次异步过程,但对用户来讲,只是一个元件。
* 用户可以方便的定义自己的元件。比如一个算法,一次通信。
* 无状态的协议实现起来极其简单。如果协议有登录,选库等过程,则会复杂一些,可参考redis实现。
* 通过强大的Upstream系统,能实现非常复杂的服务治理,例如通信节点选择,负载均衡,熔断与恢复,主备等。
* 总之这是一个企业级的,设计优雅的异步编程框架,几乎可以覆盖一切高性能的后端服务需求。
## Install CMake
#### 使用教程:
* Client基础
* [创建第一个任务:wget](docs/tutorial-01-wget.md)
* [实现一次redis写入与读出:redis_cli](docs/tutorial-02-redis_cli.md)
* [任务序列的更多功能:wget_to_redis](docs/tutorial-03-wget_to_redis.md)
* Server基础
* [第一个server:http_echo_server](docs/tutorial-04-http_echo_server.md)
* [异步server的示例:http_proxy](docs/tutorial-05-http_proxy.md)
* 并行任务与工作流 
* [一个简单的并行抓取:parallel_wget](docs/tutorial-06-parallel_wget.md)
* 几个重要的话题
* [关于错误处理](docs/about-error.md)
* [关于超时](docs/about-timeout.md)
* [关于DNS](docs/about-dns.md)
* [关于程序退出](docs/about-exit.md)
* 计算任务
* [使用内置算法工厂:sort_task](docs/tutorial-07-sort_task.md)
* [自定义计算任务:matrix_multiply](docs/tutorial-08-matrix_multiply.md)
* 文件异步IO任务
* [异步IO的http server:http_file_server](docs/tutorial-09-http_file_server.md)
* 用户定义协议基础
* [简单的用户自定义协议client/server](docs/tutorial-10-user_defined_protocol.md)
* 定时与计数任务
* [关于定时器](docs/about-timer.md)
* [关于计数器](docs/about-counter.md)
* 服务治理
* [关于服务治理](docs/about-service-management.md)
* [Upstream更多文档](docs/about-upstream.md)
* 连接上下文的使用
* [关于连接上下文](docs/about-connection-context.md)
* 内置协议用法
* [异步MySQL客户端:mysql_cli](docs/tutorial-12-mysql_cli.md)
Visit [CMake Homepage](https://cmake.org/download/), and download the msi installer to install cmake.
## Authors
## Install OpenSSL
### Ways to Install OpenSSL Library On Windos
* Download the source code to compile
* Donnload the bianary installer of OpenSSL
* Install by Chocolatey
* Install by VCPKG
### Compile OpenSSL by Source Code
Visit [OpenSSL Homepage](https://www.openssl.org/) or [Github repository](https://github.com/openssl/openssl) to download the source code.
Compile OpenSSL by its document.
### Install OpenSSL by binary installer
Google *OpenSSL binary installer* and download the installer to install. **Do not use light version**, because it doesn't contains *include and lib dictionary*.
[Here](https://slproweb.com/products/Win32OpenSSL.html) is a binary installer download website.
### Install OpenSSL by Chocolatey
[Chocolatey](https://community.chocolatey.org/) is a powerful package manager on windows. It's like apt-get or yum on some Linux distribution.
Use the command to install OpenSSL by Chocolatey.
```powershell
choco install openssl
```
### Install OpenSSL by VCPKG
[VCPKG](https://docs.microsoft.com/zh-cn/cpp/build/vcpkg?view=vs-2019) is a powerful CPP package manager across platforms.
It simplify the installation of some libraries.
Use the command to install OpenSSL by vcpkg.
```powershell
vcpkg install openssl
```
## Compile Workflow
### Compile OpenSSL by source code
Specify the **OPENSSL_ROOT_DIR** to generate Visual Studio Project, Such as
```powershell
cmake -B [build directory] -S . -DOPENSSL_ROOT_DIR=[openssl directory]
```
### Install OpenSSL by Installer or Chocolatey
Use Command to generate Visual Studio Project
```powershell
cmake -B [build directory] -S .
```
### Install OpenSSL by vcpkg
Specify **CMAKE_TOOLCHAIN_FILE** is essential to generate Visual Studio Project, Suche as
```powershell
cmake -B [build directory] -S . -DCMAKE_TOOLCHAIN_FILE=[vcpkg.cmake directory]
# If the above command fails, try to specify VCPKG_TARGET_TRIPLET, use x86-windows or x64-windows
cmake -B [build directory] -S . -DVCPKG_TARGET_TRIPLET=x86-windows -DCMAKE_TOOLCHAIN_FILE=[vcpkg.cmake directory]
```
### Others
**[openssl directory]**: openssl directory of source code
**[vcpkg.cmake directory]** : [vcpkg-root]\scripts\buildsystems\vcpkg.cmake
The VS Project(workflo.sln) will be generated now.
Use VS to open workflow.sln or use cmake command to compile workflow, Such as
```powershell
# Compile Debug Config
cmake --build [build directory] --config Debug
# Compile Release Config
cmake --build [build directory] --config Release
```
### Compile Tutorial of Workflow
```powershell
# Generate VS Project of tutorial
cmake -B build_tutorial tutorial
# Compile Debug Config
cmake --build build_tutorial --config Debug
# Compile Release Config
cmake --build build_tutorial --config Release
# compile result will be generated in tutorial\Debug or tutorial\Release directory
```
# Install Workflow by VCPKG
Workflow is already supported by VCPKG. If you don't care the source code of workflow, It's more easier to use vcpkg to install it.
[VCPKG Guide](https://docs.microsoft.com/zh-cn/cpp/build/vcpkg?view=msvc-160)
Install current version of workflow:`vcpkg install workflow`
Install HEAD version of workflow:`vcpkg install workflow --head`
Some examples:
## On Windows
```powershell
md D:\tmp
cd D:\tmp
git clone https://github.com/microsoft/vcpkg.git
cd vcpkg
.\bootstrap-vcpkg.bat
# x86-windows is default, it means .\vcpkg.exe install workflow:x86-windows
.\vcpkg.exe install workflow
# Specify x64-windows to install X64 version
.\vcpkg.exe install workflow:x64-windows
cd D:\tmp
git clone https://github.com/dengjunplusplus/workflow-vcpkg-tutorial
cd D:\tmp\workflow-vcpkg-tutorial\workflow
cmake -DCMAKE_TOOLCHAIN_FILE=D:/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -B build
# If the above command fails, try to specify VCPKG_TARGET_TRIPLET, use x86-windows or x64-windows
cmake -DCMAKE_TOOLCHAIN_FILE=D:/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x86-windows -B build
cmake --build build --config Debug
cmake --build build --config Release
```
## On Linux & Mac
```bash
cd /tmp
rm -rf vcpkg
rm -rf workflow-vcpkg-tutorial
git clone https://github.com/microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg install workflow
cd ..
git clone https://github.com/dengjunplusplus/workflow-vcpkg-tutorial.git
cd workflow-vcpkg-tutorial/workflow
cmake -DCMAKE_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -B build
# If the above command fails, try to specify VCPKG_TARGET_TRIPLET, x64-linux or x86-linux or x64-osx
cmake -DCMAKE_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-linux -B build
cmake --build build --config Debug
cmake --build build --config Release
```
# Contact Us
If you have some problems about use Workflow or compile it. Welcome to commit Issue or Email us.
## Problems about Use Workflow
* **Xie Han** - *[xiehan@sogou-inc.com](mailto:xiehan@sogou-inc.com)*
* **Wu Jiaxu** - *[wujiaxu@sogou-inc.com](mailto:wujiaxu@sogou-inc.com)*
* **Li Yingxin** - *[liyingxin@sogou-inc.com](mailto:liyingxin@sogou-inc.com)*
## Problems about Compile Workflow On Windows Or VCPKG
* **Deng Jun** - *[dengjun@sogou-inc.com](mailto:dengjun@sogou-inc.com)*
+179
View File
@@ -0,0 +1,179 @@
[Workflow 中文介绍](https://github.com/sogou/workflow/blob/master/README_cn.md)
[Windows Build Guide in English](Readme.md)
[![license MIT](https://img.shields.io/badge/License-Apache-yellow.svg)](https://git.sogou-inc.com/wujiaxu/Filter/blob/master/LICENSE)
[![C++](https://img.shields.io/badge/language-c++-red.svg)](https://en.cppreference.com/)
[![platform](https://img.shields.io/badge/platform-linux%20%7C%20macos%20%7C%20windows-lightgrey.svg)](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9)
[![Build Status](https://travis-ci.com/sogou/workflow.svg?branch=windows)](https://travis-ci.com/sogou/workflow)
# Windows下编译
Workflow采用CMake作为构建系统,依赖OpenSSL,编译workflow前需要先安装**CMake和OpenSSL**
## 安装CMake
通过[CMake官网](https://cmake.org/download/)下载cmake对应的安装包安装即可
## 安装OpenSSL
### Windows下安装OpenSSL库包括以下几种方式
* 下载源码自行编译
* 下载二进制文件或安装包
* 通过Chocolatey安装
* 通过vcpkg安装
### 下载源码自行编译
可以去官网或者GitHub仓库进行代码下载,编译过程参见OpenSSL源码中的编译文档。
* [openssl官网](https://www.openssl.org/)
* [github仓库](https://github.com/openssl/openssl)
### 下载安装包安装
请自行通过搜索引擎进行搜索,将文件解压到特定目录,或者下载并运行安装包。
安装包下载时,请**不要安装light版本**,因为light版本中不包含include和lib目录,无法正常编译
* [安装包下载](https://slproweb.com/products/Win32OpenSSL.html)
### 通过Chocolatey安装
Chocolatey是一个 Windows下的包管理器,类似于 Linux 下的 apt-get或 yum,可以方便的进行软件包的安装,Chocolatey的安装参考[Chocolatey官网](https://www.chocolatey.org/)
安装完Chocolatey后,执行下面的命令,即可安装OpenSSL
```powershell
choco install openssl
```
### 通过vcpkg安装
vcpkg是一个Windows下的C++包管理工具,它极大地简化了第三方库的配置与安装,vcpkg的安装请参考微软官方文档[VCPKG官方文档](https://docs.microsoft.com/zh-cn/cpp/build/vcpkg?view=vs-2019)
安装完vcpkg后,执行下面的命令,即可安装OpenSSL
```powershell
vcpkg install openssl
```
## 编译workflow库
### 通过源码编译OpenSSL,生成VS工程
通过源码编译或者下载二进制进行openssl安装的,在编译过程中可以通过指定OPENSSL_ROOT_DIR的方式进行编译,在workflow根目录下执行下面的命令
```powershell
cmake -B [build directory] -S . -DOPENSSL_ROOT_DIR=[openssl directory]
```
### 通过安装包或Chocolatey安装OpenSSL,生成VS工程
通过安装包安装或者通过Chocolatey安装openssl的,无需指定openssl的路径,在workflow根目录下执行下面的命令
```powershell
cmake -B [build directory] -S .
```
### 通过vcpkg安装OpenSSL,生成VS工程
通过vcpkg安装openssl的,需要指定CMAKE_TOOLCHAIN_FILE的路径
```powershell
cmake -B [build directory] -S . -DCMAKE_TOOLCHAIN_FILE=[vcpkg.cmake directory]
# 如果上面执行失败,可以尝试指定DVCPKG_TARGET_TRIPLET。 x86-windows或x64-windows
cmake -B [build directory] -S . -DVCPKG_TARGET_TRIPLET=x86-windows -DCMAKE_TOOLCHAIN_FILE=[vcpkg.cmake directory]
```
### 其他
**[openssl directory]**: openssl目录,请自行替换
**[build directory]** : 构建目录,一般使用build目录
**[vcpkg.cmake directory]** : 一般为vcpkg根目录\scripts\buildsystems\vcpkg.cmake
执行完成后,在[build directory]下会生成workflow.sln,可以用VS打开进行编译,或执行cmake命令进行编译
```powershell
# 编译Debug版本
cmake --build [build directory] --config Debug
# 编译Release版本
cmake --build [build directory] --config Release
```
### 编译tutorial
```powershell
# 生成tutorial工程
cmake -B build_tutorial tutorial
# 编译Debug版本
cmake --build build_tutorial --config Debug
# 编译Release版本
cmake --build build_tutorial --config Release
# tutorial\Debug或者tutorial\Release目录下即为tutorial编译结果
```
# VCPKG安装workflow
Workflow已经支持VCPKG安装啦!**如果只是使用workflow,而不关注源码,可以使用VCPKG安装**。
VCPKG如何安装及使用详见[官方文档](https://docs.microsoft.com/zh-cn/cpp/build/vcpkg?view=msvc-160)
安装workflow当前版本:`vcpkg install workflow`
安装workflow最新HEAD提交版本:`vcpkg install workflow --head`
下面是各平台的安装示例:
## On Windows
```powershell
md D:\tmp
cd D:\tmp
git clone https://github.com/microsoft/vcpkg.git
cd vcpkg
.\bootstrap-vcpkg.bat
# 默认安装x86-windows,与.\vcpkg.exe install workflow:x86-windows相同
.\vcpkg.exe install workflow
# 也可以指定安装x64
.\vcpkg.exe install workflow:x64-windows
cd D:\tmp
git clone https://github.com/dengjunplusplus/workflow-vcpkg-tutorial
cd D:\tmp\workflow-vcpkg-tutorial\workflow
cmake -DCMAKE_TOOLCHAIN_FILE=D:/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -B build
# 如果上面执行失败,可以尝试指定VCPKG_TARGET_TRIPLET。 x86-windows或x64-windows
cmake -DCMAKE_TOOLCHAIN_FILE=D:/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x86-windows -B build
cmake --build build --config Debug
cmake --build build --config Release
```
## On Linux & Mac
```bash
cd /tmp
rm -rf vcpkg
rm -rf workflow-vcpkg-tutorial
git clone https://github.com/microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg install workflow
cd ..
git clone https://github.com/dengjunplusplus/workflow-vcpkg-tutorial.git
cd workflow-vcpkg-tutorial/workflow
cmake -DCMAKE_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -B build
# 如果上面执行失败,可以尝试指定DVCPKG_TARGET_TRIPLET。 x64-linux 或x86-linux或x64-osx
cmake -DCMAKE_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-linux -B build
cmake --build build --config Debug
cmake --build build --config Release
```
# 联系方式
如果编译和使用过程有什么问题,欢迎提Issue,或者通过邮件和QQ群(618773193)联系我们
## Workflow使用相关问题
* **Xie Han** - *[xiehan@sogou-inc.com](mailto:xiehan@sogou-inc.com)*
* **Li Yingxin** - *[liyingxin@sogou-inc.com](mailto:liyingxin@sogou-inc.com)*
## Windows分支相关问题或VCPKG使用问题
* **Deng Jun** - *[dengjun@sogou-inc.com](mailto:dengjun@sogou-inc.com)*
-112
View File
@@ -1,112 +0,0 @@
[中文版](README.md)
[![license MIT](https://img.shields.io/badge/License-Apache-yellow.svg)](https://git.sogou-inc.com/wujiaxu/Filter/blob/master/LICENSE)
[![C++](https://img.shields.io/badge/language-c++-red.svg)](https://en.cppreference.com/)
[![platform](https://img.shields.io/badge/platform-linux%20%7C%20macos-lightgrey.svg)](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9)
# Sogou C++ Workflow
#### As the backend C++ programming standard in Sogou, Workflow is an industrial-grade programming engine.
#### Main functions and features:
* An **asynchronous engine** based on **C++11** ``std::function`` which aims to solve all the **serial, parallel and asynchronous** problems.
* As a network framework, it is completely **protocol-agnostic** and directly facing applications.
* It can either be used as a Redis **client** or an Http **server**.
* Convinient to **customize protocols**, so you can quickly build your own RPC systems.
* Sogou RPC is developed based on Sogou Workflow and is open source as an independent project. The project supports srpc, brpc and thrift protocols ([benchmark](https://github.com/holmes1412/sogou-rpc-benchmark)).
* Support **SSL** (depends on openssl). Support **TCP, UDP, SCTP** and other common transport layer protocols. Support SSL on **SCTP**. Not support UDP server.
* A variety of **common Internet protocol** implementations are natively integrated and used in a unified way.
* Currently support **http, redis, mysql** and **kafka** protocols. You can directly access these resources or build **servers** for these protocols.
* Highly likely the only C++ full-featured **mysql asynchronous client** on the market.
* **DNS** protocol is being developing and currently we use the system library to access DNS.
* Powerful feature for **scheduling computing tasks**
* Computing task, as well as communication task, can be added into the task flow and they’re scheduled separately by their corresponding scheduler.
* You can use it as a parallel programming engine **without** the network features.
* Our biggest goal is to **maximize the performance** of every node when the calculation and communication environment is very complex.
* Some **common algorithm** implementations are provided, such as parallel sorting and MapReduce.
* In fact, all asynchronous processes (such as disk IO, GPU tasks, timers, etc.) **can be scheduled in coordination**.
* On the Linux system, the disk IO task is realized through the Linux underlying aio, which is extremely efficient.
* Support any task flow with **DAG** structure. However, in most cases, users only need **series-parallel** structure.
* Built-in **load balancing** and powerful **service governance** features.
* Easily used in conjunction with other asynchronous engines.
* **Streaming** communication engine is being developed.
* When working as a server, it supports **multi-processes** mode and precisely supports **graceful restart**.
#### Building
* Support **Linux, macOS, FreeBSD, Windows** and other systems so far. Installing **cmake** is necessary.
* Windows version is temporarily released as an independent branch, which uses **iocp** as the basis for asynchronous communication and mean while, keeping the **same external interface**.
* As written in C/C++, it requires the users being able to proficiently use C++ programing. It **does not** rely on boost or asio, therefor the compiling speed is extremely fast.
* It contains a small numbers of C++11 features, so users need to being able to use ``std::function`` and ``std::move``.
* Theoretically support all CPU architectures and can be compiled and run on **32-bit** or **64-bit arm processors**. Big endian CPU is not tested.
* **openssl** is required. If users expect high performance of SSL, openssl 1.1 or higher is strongly recommended.
* **No other dependencies**. Several compression libraries such as snappy and lz4 is contained by their unmodified source (required by the Kafka protocol).
#### Some features of design
* The basic usage is very simple and handy. Some features are designed to greatly reduce the difficulty of programming with general C++ projects.
* To **avoid** users to **derive** as much as possible, all user behaviors are wrapped with std::function, for example:
* the **callback** after every task ends
* the **algorithms** in computing tasks
* one server corresponds to one ``std::function``
* Trying to avoid complicated memory management, all tasks and frameworks are generated by **factory** classes, and their memory is **recycled automatically**. Which means,
* Every task is **automatically deleted after its callback**.
* If the users want to keep any data in the task (such as a network reply packet or the result of an algorithm), they need to use ``std::move`` to move it.
* We treat memory recycle as a strict and naturally logical mechanism, so we **don’t** use share_ptr.
* Avoid using complicated parameter configuration.
* Actually we have a lot of **configurable parameters**, though you can use our system **without** feeling the parameters exist.
* If you have specific requirements for program behavior and resource ratio, you can definitely find the corresponding parameter configuration items in order to maximize the performance of you program.
* The project adopts a fully asynchronous design and is not transparent to users, which means users need to know that they are writing asynchronous programs.
* Thanks to the convenience brought by ``std::function`` and the automatic memory recycling mechanism, we have delicately designed **the simply possible usage of asynchrony** for users.
* **No** user-mode threads concepts. On the one hand, performance is considered. On the other hand, we have the concept of computing tasks (threaded tasks) scheduling.
* In our design, **computing** is one kind of **asynchronous task**, which has no differences from communication.
* Computing tasks are scheduled by **independent thread groups** according to specific algorithms, please note that they **may not** be executed **immediately**.
* As we have such computing tasks, user-mode threads become meaningless, and therefore users must understand asynchrony.
* Because of the full asynchrony, almost all core calls are **short** and **non-blocking** operations.
* That’s why we **don’t** recommend users to **block** their programs in callback or do some complex calculations. However, it acceptable if the logic is quite simple.
* Brief summary of the usage:
* The user builds the program just like building a **series-parallel** circuit. The circuit can be generated **at the beginning** or **dynamically during the program running**.
* **We provide various electronic components** for users. For instance, one http request, one GPU matrix multiplication, and one parallel sorting can all be understood as a electronic component.
* Every electronic component has its **standard input and output**. At the meantime, every electronic component can be a **complicated circuit**, which has no necessary to be perceived by the users.
* For example, an http request may go through **multiple asynchronous processes** such as DNS, redirect, and retry, but the entire processes is just a **component** in the perspective of the users.
* Users can easily **define their own** components, including algorithms and some kind of communication.
* To implement **stateless protocols** is extremely simple. It may be a little bit complicated when the protocol includes login, library selection, etc., at this time, you can refer to the redis implementation.
* Through the powerful Upstream system, complex **service governance** can be realized, such as communication node selection, load balancing, circuit breaker and recovery, main and backup, etc.
* **In conclusion, this is an enterprise-level, elegantly designed asynchronous framework which can cover almost all high-performance back-end service requirements.**
#### Tutorials:
* Client
* [Create your first task:wget](docs/tutorial-01-wget.md)
* [Implement redis set and get:redis_cli](docs/tutorial-02-redis_cli.md)
* [More features about series:wget_to_redis](docs/tutorial-03-wget_to_redis.md)
* Server
* [first server:http_echo_server](docs/tutorial-04-http_echo_server.md)
* [asynchronous server:http_proxy](docs/tutorial-05-http_proxy.md)
* Paralle task and Series 
* [A simple parallel wget:parallel_wget](docs/tutorial-06-parallel_wget.md)
* Important topic
* [About error](docs/about-error.md)
* [About timeout](docs/about-timeout.md)
* [About DNS](docs/about-dns.md)
* [About exit](docs/about-exit.md)
* Computing tasks
* [Using the build-in algorithm factory:sort_task](docs/tutorial-07-sort_task.md)
* [User-define computing task:matrix_multiply](docs/tutorial-08-matrix_multiply.md)
* File asynchronous IO tasks
* [Http server with file IO:http_file_server](docs/tutorial-09-http_file_server.md)
* User-defined protocol basic usage
* [A simple user-defined portocol client/server](docs/tutorial-10-user_defined_protocol.md)
* Timing tasks and counting tasks
* [About timer](docs/about-timer.md)
* [About counter](docs/about-counter.md)
* Service governance
* [About service governance](docs/about-service-management.md)
* [More documents about upstream](docs/about-upstream.md)
* Connection context
* [About connection context](docs/about-connection-context.md)
* Built-in protocols
* [Asynchronous MySQL client:mysql_cli](docs/tutorial-12-mysql_cli.md)
## Authors
* **Xie Han** - *[xiehan@sogou-inc.com](mailto:xiehan@sogou-inc.com)*
* **Wu Jiaxu** - *[wujiaxu@sogou-inc.com](mailto:wujiaxu@sogou-inc.com)*
* **Li Yingxin** - *[liyingxin@sogou-inc.com](mailto:liyingxin@sogou-inc.com)*