From fc3ad2eb74cf1249d092eb5132b4dccc5c07f891 Mon Sep 17 00:00:00 2001 From: DengJun Date: Sat, 31 Jul 2021 21:44:37 +0800 Subject: [PATCH] update README of windows branch --- README.md | 271 ++++++++++++++++++++++++++++++++------------------- README_cn.md | 179 ++++++++++++++++++++++++++++++++++ README_en.md | 112 --------------------- 3 files changed, 352 insertions(+), 210 deletions(-) create mode 100644 README_cn.md delete mode 100644 README_en.md diff --git a/README.md b/README.md index 5a7cb1e7..2812edec 100644 --- a/README.md +++ b/README.md @@ -1,112 +1,187 @@ -[English version](README_en.md) +[Workflow Introduction](https://github.com/sogou/workflow/blob/master/README.md) + +[Windows下Workflow编译指南](Readme_cn.md) [![license MIT](https://img.shields.io/badge/License-Apache-yellow.svg)](https://git.sogou-inc.com/wujiaxu/Filter/blob/master/LICENSE) [![C++](https://img.shields.io/badge/language-c++-red.svg)](https://en.cppreference.com/) -[![platform](https://img.shields.io/badge/platform-linux%20%7C%20macos-lightgrey.svg)](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9) +[![platform](https://img.shields.io/badge/platform-linux%20%7C%20macos%20%7C%20windows-lightgrey.svg)](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9) [![Build Status](https://travis-ci.com/sogou/workflow.svg?branch=windows)](https://travis-ci.com/sogou/workflow) -# Sogou C++ Workflow -#### 搜狗公司的后端C++编程标准，是一套企业级的程序引擎。主要功能和特点： - * 这是一个基于C++11 ``std::function`` 的异步引擎。用于解决一切关于串行，并行和异步的问题。 - * 作为网络框架，完全协议无关，并且直接面向应用。 - * 可以当作一个异步redis客户端，也可以快速搭建一个Http服务器。 - * 自定义协议非常方便，你可以快速的建立一个自己的RPC系统。 - * Sogou RPC就是以它为基础开发，作为独立项目开源。该项目支持srpc，brpc和thrift等协议（[benchmark](https://github.com/holmes1412/sogou-rpc-benchmark)）。 - * 支持SSL(依赖openssl)。支持TCP, UDP，SCTP等常用传输层协议。可支持SCTP上的SSL。但不支持UDP server。 - * 原生包含了多种常有互联网协议的实现，并且以统一的方式使用。 - * 目前支持http，redis，mysql和kafka协议，可以直接访问这些资源或搭建这些协议的server。 - * 可能是目前市面上唯一C++全功能的mysql异步客户端。 - * DNS协议开发中，目前使用系统库访问dns。 - * 包含强大的计算任务调度功能。 - * 计算任务可以和通信任务一样都可以加入任务流，有各自的调度器实现调度。 - * 你可以不用网络功能，把它当并行编程工具来使用。 - * 我们最大的目的就是在计算和通信非常复杂的情况下，最大程度的发挥节点的性能。 - * 提供了一些常用算法实现，例如并行排序，MapReduce。 - * 实际上，一切异步过程（如磁盘IO，GPU任务，定时器等）都可以被协同调度。 - * 如果用户使用Linux系统，磁盘IO任务是是通过Linux底层aio实现，效率极高。 - * 支持任何DAG结构的任务流。但大多数情况下，用户只需要使用串并联结构。 - * 自带负载均衡和强大的服务治理等功能。 - * 可以方便的与其他异步引擎协同使用。 - * 支持streaming的通讯引擎开发中。 - * 作为server使用时，支持多进程模式，支持精确的优雅重启。 +# Build Workflow On Windows -#### 相关的技术要点： - * 目前项目支持Linux，macOS，FreeBSD，Windows等系统。需要安装cmake。 - * Windows版暂时以独立[branch](https://github.com/sogou/workflow/tree/windows)发布，以iocp为异步通讯基础，对外接口一致。 - * 该项目使用C/C++编写，使用者需要能比较熟练的使用C++编程。不依赖boost或asio，编译速度极快。 - * 用到少量的C++11特征，用户需要会使用 ``std::function`` 和 ``std::move()`` 。 - * 理论支持一切CPU架构，可以在32位或64位arm处理器上编译运行。Big endian CPU未测试。 - * 项目需要依赖openssl。如果用户对SSL性能要求高，强烈建议使用openssl 1.1以上版本。 - * 项目无其他依赖，但包含了snappy，lz4等几个压缩库的无改动源代码（kafka协议需要）。 +We use cmake to build the workflow project. It denpends one OpenSSL. It's essential to install **CMake and OpenSSL**. -#### 项目的一些设计特点： - * 项目基础用法非常简单，极易上手。而我们的一些设计也大幅降低了一般C++项目的使用难度。 - * 尽量避免让用户派生，一切用户行为都用 ``std::function`` 来包装，例如： - * 任何任务结束后的callback。 - * 计算任务里的算法。 - * 一个server，也对应一个 ``std::function`` 。 - * 尽量避免复杂的内存管理，一切任务和框架由工厂类产生，并自动内存回收。 - * 任何任务在callback之后被自动delete。 - * 任务中的数据（例如一个网络回复包，一个算法的结果），如果用户要保留，需要用 ``std::move()`` 把它移走。 - * 内存回收是使用一套严密又符合自然逻辑的机制，我们不使用任何share_ptr。 - * 尽量避免复杂的参数配置。 - * 我们可配置的参数非常多，但您可以在对参数完全无感的情况下使用我们的系统。 - * 如果您对程序行为，资源配比有特定需求，一定能找到对应配置项，以便发挥程序最高性能。 - * 项目采用全异步的设计，并且对用户不透明，用户需要知道自己在写异步程序。 - * 我们通过精心的设计的用户尽量简单的使用异步，得益于 ``std::function`` 带来的便利性，以及自动内存回收机制。 - * 没有采用用户态线程的方式，一方面是性能考虑，另一方面原因是我们有计算任务（线程任务）调度的概念。 - * 在我们的设计里，计算是一种异步任务，和通信并没有什么区别。 - * 计算任务由独立的线程组依照特定算法进行调度，并不一定实时被执行。 - * 因为有这种计算任务的存在，用户态线程似乎就没什么意义了，用户必然需要了解异步。 - * 因为全异步的原因，我们几乎所有的核心调用都是短操作，不会阻塞。 - * 我们也不鼓励用户在callback里阻塞程序，或做复杂的计算。但如果业务逻辑简单，这么做也无妨。 - * 项目使用方法的简单总结： - * 用户如同搭建串并联电路一样搭建程序。电路可以一开始就生成好，也可以运行中动态产生。 - * 我们提供各种电子元件供用户使用，一次http请求，一次GPU矩阵乘法，一次并行排序，都可以理解成为一个元件。 - * 每个电子元件都有标准的输入输出。但每个电子元件内部，可能又是一个复杂电路，但用户并不感知。 - * 比如一个http请求，可能要经历DNS，redirect，retry等多次异步过程，但对用户来讲，只是一个元件。 - * 用户可以方便的定义自己的元件。比如一个算法，一次通信。 - * 无状态的协议实现起来极其简单。如果协议有登录，选库等过程，则会复杂一些，可参考redis实现。 - * 通过强大的Upstream系统，能实现非常复杂的服务治理，例如通信节点选择，负载均衡，熔断与恢复，主备等。 - * 总之这是一个企业级的，设计优雅的异步编程框架，几乎可以覆盖一切高性能的后端服务需求。 +## Install CMake -#### 使用教程： - * Client基础 - * [创建第一个任务：wget](docs/tutorial-01-wget.md) - * [实现一次redis写入与读出：redis_cli](docs/tutorial-02-redis_cli.md) - * [任务序列的更多功能：wget_to_redis](docs/tutorial-03-wget_to_redis.md) - * Server基础 - * [第一个server：http_echo_server](docs/tutorial-04-http_echo_server.md) - * [异步server的示例：http_proxy](docs/tutorial-05-http_proxy.md) - * 并行任务与工作流　 - * [一个简单的并行抓取：parallel_wget](docs/tutorial-06-parallel_wget.md) - * 几个重要的话题 - * [关于错误处理](docs/about-error.md) - * [关于超时](docs/about-timeout.md) - * [关于DNS](docs/about-dns.md) - * [关于程序退出](docs/about-exit.md) - * 计算任务 - * [使用内置算法工厂：sort_task](docs/tutorial-07-sort_task.md) - * [自定义计算任务：matrix_multiply](docs/tutorial-08-matrix_multiply.md) - * 文件异步IO任务 - * [异步IO的http server：http_file_server](docs/tutorial-09-http_file_server.md) - * 用户定义协议基础 - * [简单的用户自定义协议client/server](docs/tutorial-10-user_defined_protocol.md) - * 定时与计数任务 - * [关于定时器](docs/about-timer.md) - * [关于计数器](docs/about-counter.md) - * 服务治理 - * [关于服务治理](docs/about-service-management.md) - * [Upstream更多文档](docs/about-upstream.md) - * 连接上下文的使用 - * [关于连接上下文](docs/about-connection-context.md) - * 内置协议用法 - * [异步MySQL客户端：mysql_cli](docs/tutorial-12-mysql_cli.md) +Visit [CMake Homepage](https://cmake.org/download/), and download the msi installer to install cmake. -## Authors +## Install OpenSSL + +### Ways to Install OpenSSL Library On Windos + +* Download the source code to compile +* Donnload the bianary installer of OpenSSL +* Install by Chocolatey +* Install by VCPKG + +### Compile OpenSSL by Source Code + +Visit [OpenSSL Homepage](https://www.openssl.org/) or [Github repository](https://github.com/openssl/openssl) to download the source code. +Compile OpenSSL by its document. + +### Install OpenSSL by binary installer + +Google *OpenSSL binary installer* and download the installer to install. **Do not use light version**, because it doesn't contains *include and lib dictionary*. + +[Here](https://slproweb.com/products/Win32OpenSSL.html) is a binary installer download website. + + +### Install OpenSSL by Chocolatey + +[Chocolatey](https://community.chocolatey.org/) is a powerful package manager on windows. It's like apt-get or yum on some Linux distribution. + +Use the command to install OpenSSL by Chocolatey. + +```powershell +choco install openssl +``` + +### Install OpenSSL by VCPKG + +[VCPKG](https://docs.microsoft.com/zh-cn/cpp/build/vcpkg?view=vs-2019) is a powerful CPP package manager across platforms. +It simplify the installation of some libraries. + +Use the command to install OpenSSL by vcpkg. + +```powershell +vcpkg install openssl +``` + +## Compile Workflow +### Compile OpenSSL by source code + +Specify the **OPENSSL_ROOT_DIR** to generate Visual Studio Project, Such as + +```powershell +cmake -B [build directory] -S . -DOPENSSL_ROOT_DIR=[openssl directory] +``` + +### Install OpenSSL by Installer or Chocolatey + +Use Command to generate Visual Studio Project + +```powershell +cmake -B [build directory] -S . +``` + +### Install OpenSSL by vcpkg +Specify **CMAKE_TOOLCHAIN_FILE** is essential to generate Visual Studio Project, Suche as + +```powershell + cmake -B [build directory] -S . -DCMAKE_TOOLCHAIN_FILE=[vcpkg.cmake directory] +# If the above command fails, try to specify VCPKG_TARGET_TRIPLET, use x86-windows or x64-windows +cmake -B [build directory] -S . -DVCPKG_TARGET_TRIPLET=x86-windows -DCMAKE_TOOLCHAIN_FILE=[vcpkg.cmake directory] +``` + +### Others +**[openssl directory]**: openssl directory of source code + +**[vcpkg.cmake directory]** : [vcpkg-root]\scripts\buildsystems\vcpkg.cmake + + +The VS Project(workflo.sln) will be generated now. + +Use VS to open workflow.sln or use cmake command to compile workflow, Such as + +```powershell +# Compile Debug Config +cmake --build [build directory] --config Debug +# Compile Release Config +cmake --build [build directory] --config Release +``` + +### Compile Tutorial of Workflow +```powershell + +# Generate VS Project of tutorial +cmake -B build_tutorial tutorial + +# Compile Debug Config +cmake --build build_tutorial --config Debug +# Compile Release Config +cmake --build build_tutorial --config Release + +# compile result will be generated in tutorial\Debug or tutorial\Release directory +``` + +# Install Workflow by VCPKG + +Workflow is already supported by VCPKG. If you don't care the source code of workflow, It's more easier to use vcpkg to install it. + +[VCPKG Guide](https://docs.microsoft.com/zh-cn/cpp/build/vcpkg?view=msvc-160) + +Install current version of workflow：`vcpkg install workflow` + +Install HEAD version of workflow：`vcpkg install workflow --head` + +Some examples: + +## On Windows + +```powershell +md D:\tmp +cd D:\tmp +git clone https://github.com/microsoft/vcpkg.git +cd vcpkg +.\bootstrap-vcpkg.bat +# x86-windows is default, it means .\vcpkg.exe install workflow:x86-windows +.\vcpkg.exe install workflow +# Specify x64-windows to install X64 version +.\vcpkg.exe install workflow:x64-windows + +cd D:\tmp +git clone https://github.com/dengjunplusplus/workflow-vcpkg-tutorial + +cd D:\tmp\workflow-vcpkg-tutorial\workflow +cmake -DCMAKE_TOOLCHAIN_FILE=D:/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -B build +# If the above command fails, try to specify VCPKG_TARGET_TRIPLET, use x86-windows or x64-windows +cmake -DCMAKE_TOOLCHAIN_FILE=D:/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x86-windows -B build +cmake --build build --config Debug +cmake --build build --config Release + +``` + +## On Linux & Mac + +```bash +cd /tmp +rm -rf vcpkg +rm -rf workflow-vcpkg-tutorial +git clone https://github.com/microsoft/vcpkg.git +cd vcpkg +./bootstrap-vcpkg.sh +./vcpkg install workflow + +cd .. +git clone https://github.com/dengjunplusplus/workflow-vcpkg-tutorial.git +cd workflow-vcpkg-tutorial/workflow +cmake -DCMAKE_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -B build +# If the above command fails, try to specify VCPKG_TARGET_TRIPLET, x64-linux or x86-linux or x64-osx +cmake -DCMAKE_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-linux -B build +cmake --build build --config Debug +cmake --build build --config Release + +``` + +# Contact Us + +If you have some problems about use Workflow or compile it. Welcome to commit Issue or Email us. + +## Problems about Use Workflow * **Xie Han** - *[xiehan@sogou-inc.com](mailto:xiehan@sogou-inc.com)* -* **Wu Jiaxu** - *[wujiaxu@sogou-inc.com](mailto:wujiaxu@sogou-inc.com)* * **Li Yingxin** - *[liyingxin@sogou-inc.com](mailto:liyingxin@sogou-inc.com)* +## Problems about Compile Workflow On Windows Or VCPKG +* **Deng Jun** - *[dengjun@sogou-inc.com](mailto:dengjun@sogou-inc.com)* \ No newline at end of file diff --git a/README_cn.md b/README_cn.md new file mode 100644 index 00000000..2d45dab5 --- /dev/null +++ b/README_cn.md @@ -0,0 +1,179 @@ +[Workflow 中文介绍](https://github.com/sogou/workflow/blob/master/README_cn.md) + +[Windows Build Guide in English](Readme.md) + +[![license MIT](https://img.shields.io/badge/License-Apache-yellow.svg)](https://git.sogou-inc.com/wujiaxu/Filter/blob/master/LICENSE) +[![C++](https://img.shields.io/badge/language-c++-red.svg)](https://en.cppreference.com/) +[![platform](https://img.shields.io/badge/platform-linux%20%7C%20macos%20%7C%20windows-lightgrey.svg)](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9) +[![Build Status](https://travis-ci.com/sogou/workflow.svg?branch=windows)](https://travis-ci.com/sogou/workflow) + +# Windows下编译 + +Workflow采用CMake作为构建系统，依赖OpenSSL，编译workflow前需要先安装**CMake和OpenSSL** + +## 安装CMake +通过[CMake官网](https://cmake.org/download/)下载cmake对应的安装包安装即可 + +## 安装OpenSSL + +### Windows下安装OpenSSL库包括以下几种方式 + +* 下载源码自行编译 +* 下载二进制文件或安装包 +* 通过Chocolatey安装 +* 通过vcpkg安装 + +### 下载源码自行编译 + +可以去官网或者GitHub仓库进行代码下载，编译过程参见OpenSSL源码中的编译文档。 + +* [openssl官网](https://www.openssl.org/) +* [github仓库](https://github.com/openssl/openssl) + +### 下载安装包安装 + +请自行通过搜索引擎进行搜索，将文件解压到特定目录，或者下载并运行安装包。 +安装包下载时，请**不要安装light版本**，因为light版本中不包含include和lib目录，无法正常编译 +* [安装包下载](https://slproweb.com/products/Win32OpenSSL.html) + + +### 通过Chocolatey安装 + +Chocolatey是一个 Windows下的包管理器，类似于 Linux 下的 apt-get或 yum，可以方便的进行软件包的安装，Chocolatey的安装参考[Chocolatey官网](https://www.chocolatey.org/) +安装完Chocolatey后，执行下面的命令，即可安装OpenSSL +```powershell +choco install openssl +``` + +### 通过vcpkg安装 + +vcpkg是一个Windows下的C++包管理工具，它极大地简化了第三方库的配置与安装，vcpkg的安装请参考微软官方文档[VCPKG官方文档](https://docs.microsoft.com/zh-cn/cpp/build/vcpkg?view=vs-2019) +安装完vcpkg后，执行下面的命令，即可安装OpenSSL +```powershell +vcpkg install openssl +``` + +## 编译workflow库 +### 通过源码编译OpenSSL，生成VS工程 +通过源码编译或者下载二进制进行openssl安装的，在编译过程中可以通过指定OPENSSL_ROOT_DIR的方式进行编译，在workflow根目录下执行下面的命令 + +```powershell +cmake -B [build directory] -S . -DOPENSSL_ROOT_DIR=[openssl directory] +``` + +### 通过安装包或Chocolatey安装OpenSSL，生成VS工程 +通过安装包安装或者通过Chocolatey安装openssl的，无需指定openssl的路径，在workflow根目录下执行下面的命令 + +```powershell +cmake -B [build directory] -S . +``` + +### 通过vcpkg安装OpenSSL，生成VS工程 +通过vcpkg安装openssl的，需要指定CMAKE_TOOLCHAIN_FILE的路径 + +```powershell + cmake -B [build directory] -S . -DCMAKE_TOOLCHAIN_FILE=[vcpkg.cmake directory] + # 如果上面执行失败，可以尝试指定DVCPKG_TARGET_TRIPLET。 x86-windows或x64-windows +cmake -B [build directory] -S . -DVCPKG_TARGET_TRIPLET=x86-windows -DCMAKE_TOOLCHAIN_FILE=[vcpkg.cmake directory] +``` + +### 其他 +**[openssl directory]**: openssl目录，请自行替换 + +**[build directory]** : 构建目录，一般使用build目录 + +**[vcpkg.cmake directory]** : 一般为vcpkg根目录\scripts\buildsystems\vcpkg.cmake + +执行完成后，在[build directory]下会生成workflow.sln，可以用VS打开进行编译，或执行cmake命令进行编译 + +```powershell +# 编译Debug版本 +cmake --build [build directory] --config Debug +# 编译Release版本 +cmake --build [build directory] --config Release +``` + +### 编译tutorial +```powershell + +# 生成tutorial工程 +cmake -B build_tutorial tutorial + +# 编译Debug版本 +cmake --build build_tutorial --config Debug +# 编译Release版本 +cmake --build build_tutorial --config Release + +# tutorial\Debug或者tutorial\Release目录下即为tutorial编译结果 +``` + +# VCPKG安装workflow + +Workflow已经支持VCPKG安装啦！**如果只是使用workflow，而不关注源码，可以使用VCPKG安装**。 + +VCPKG如何安装及使用详见[官方文档](https://docs.microsoft.com/zh-cn/cpp/build/vcpkg?view=msvc-160) + +安装workflow当前版本：`vcpkg install workflow` + +安装workflow最新HEAD提交版本：`vcpkg install workflow --head` + +下面是各平台的安装示例： + +## On Windows + +```powershell +md D:\tmp +cd D:\tmp +git clone https://github.com/microsoft/vcpkg.git +cd vcpkg +.\bootstrap-vcpkg.bat +# 默认安装x86-windows，与.\vcpkg.exe install workflow:x86-windows相同 +.\vcpkg.exe install workflow +# 也可以指定安装x64 +.\vcpkg.exe install workflow:x64-windows + +cd D:\tmp +git clone https://github.com/dengjunplusplus/workflow-vcpkg-tutorial + +cd D:\tmp\workflow-vcpkg-tutorial\workflow +cmake -DCMAKE_TOOLCHAIN_FILE=D:/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -B build +# 如果上面执行失败，可以尝试指定VCPKG_TARGET_TRIPLET。 x86-windows或x64-windows +cmake -DCMAKE_TOOLCHAIN_FILE=D:/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x86-windows -B build +cmake --build build --config Debug +cmake --build build --config Release + +``` + +## On Linux & Mac + +```bash +cd /tmp +rm -rf vcpkg +rm -rf workflow-vcpkg-tutorial +git clone https://github.com/microsoft/vcpkg.git +cd vcpkg +./bootstrap-vcpkg.sh +./vcpkg install workflow + +cd .. +git clone https://github.com/dengjunplusplus/workflow-vcpkg-tutorial.git +cd workflow-vcpkg-tutorial/workflow +cmake -DCMAKE_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -B build +# 如果上面执行失败，可以尝试指定DVCPKG_TARGET_TRIPLET。 x64-linux 或x86-linux或x64-osx +cmake -DCMAKE_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-linux -B build +cmake --build build --config Debug +cmake --build build --config Release + +``` + +# 联系方式 +如果编译和使用过程有什么问题，欢迎提Issue，或者通过邮件和QQ群(618773193)联系我们 + +## Workflow使用相关问题 + +* **Xie Han** - *[xiehan@sogou-inc.com](mailto:xiehan@sogou-inc.com)* +* **Li Yingxin** - *[liyingxin@sogou-inc.com](mailto:liyingxin@sogou-inc.com)* + +## Windows分支相关问题或VCPKG使用问题 + +* **Deng Jun** - *[dengjun@sogou-inc.com](mailto:dengjun@sogou-inc.com)* \ No newline at end of file diff --git a/README_en.md b/README_en.md deleted file mode 100644 index cd21d7a1..00000000 --- a/README_en.md +++ /dev/null @@ -1,112 +0,0 @@ -[中文版](README.md) - -[![license MIT](https://img.shields.io/badge/License-Apache-yellow.svg)](https://git.sogou-inc.com/wujiaxu/Filter/blob/master/LICENSE) -[![C++](https://img.shields.io/badge/language-c++-red.svg)](https://en.cppreference.com/) -[![platform](https://img.shields.io/badge/platform-linux%20%7C%20macos-lightgrey.svg)](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9) - -# Sogou C++ Workflow -#### As the backend C++ programming standard in Sogou, Workflow is an industrial-grade programming engine. -#### Main functions and features: - * An **asynchronous engine** based on **C++11** ``std::function`` which aims to solve all the **serial, parallel and asynchronous** problems. - * As a network framework, it is completely **protocol-agnostic** and directly facing applications. - * It can either be used as a Redis **client** or an Http **server**. - * Convinient to **customize protocols**, so you can quickly build your own RPC systems. - * Sogou RPC is developed based on Sogou Workflow and is open source as an independent project. The project supports srpc, brpc and thrift protocols ([benchmark](https://github.com/holmes1412/sogou-rpc-benchmark)). - * Support **SSL** (depends on openssl). Support **TCP, UDP, SCTP** and other common transport layer protocols. Support SSL on **SCTP**. Not support UDP server. - * A variety of **common Internet protocol** implementations are natively integrated and used in a unified way. - * Currently support **http, redis, mysql** and **kafka** protocols. You can directly access these resources or build **servers** for these protocols. - * Highly likely the only C++ full-featured **mysql asynchronous client** on the market. - * **DNS** protocol is being developing and currently we use the system library to access DNS. - * Powerful feature for **scheduling computing tasks** - * Computing task, as well as communication task, can be added into the task flow and they’re scheduled separately by their corresponding scheduler. - * You can use it as a parallel programming engine **without** the network features. - * Our biggest goal is to **maximize the performance** of every node when the calculation and communication environment is very complex. - * Some **common algorithm** implementations are provided, such as parallel sorting and MapReduce. - * In fact, all asynchronous processes (such as disk IO, GPU tasks, timers, etc.) **can be scheduled in coordination**. - * On the Linux system, the disk IO task is realized through the Linux underlying aio, which is extremely efficient. - * Support any task flow with **DAG** structure. However, in most cases, users only need **series-parallel** structure. - * Built-in **load balancing** and powerful **service governance** features. - * Easily used in conjunction with other asynchronous engines. - * **Streaming** communication engine is being developed. - * When working as a server, it supports **multi-processes** mode and precisely supports **graceful restart**. - -#### Building - * Support **Linux, macOS, FreeBSD, Windows** and other systems so far. Installing **cmake** is necessary. - * Windows version is temporarily released as an independent branch, which uses **iocp** as the basis for asynchronous communication and mean while, keeping the **same external interface**. - * As written in C/C++, it requires the users being able to proficiently use C++ programing. It **does not** rely on boost or asio, therefor the compiling speed is extremely fast. - * It contains a small numbers of C++11 features, so users need to being able to use ``std::function`` and ``std::move``. - * Theoretically support all CPU architectures and can be compiled and run on **32-bit** or **64-bit arm processors**. Big endian CPU is not tested. - * **openssl** is required. If users expect high performance of SSL, openssl 1.1 or higher is strongly recommended. - * **No other dependencies**. Several compression libraries such as snappy and lz4 is contained by their unmodified source (required by the Kafka protocol). - -#### Some features of design - * The basic usage is very simple and handy. Some features are designed to greatly reduce the difficulty of programming with general C++ projects. - * To **avoid** users to **derive** as much as possible, all user behaviors are wrapped with std::function, for example: - * the **callback** after every task ends - * the **algorithms** in computing tasks - * one server corresponds to one ``std::function`` - * Trying to avoid complicated memory management, all tasks and frameworks are generated by **factory** classes, and their memory is **recycled automatically**. Which means, - * Every task is **automatically deleted after its callback**. - * If the users want to keep any data in the task (such as a network reply packet or the result of an algorithm), they need to use ``std::move`` to move it. - * We treat memory recycle as a strict and naturally logical mechanism, so we **don’t** use share_ptr. - * Avoid using complicated parameter configuration. - * Actually we have a lot of **configurable parameters**, though you can use our system **without** feeling the parameters exist. - * If you have specific requirements for program behavior and resource ratio, you can definitely find the corresponding parameter configuration items in order to maximize the performance of you program. - * The project adopts a fully asynchronous design and is not transparent to users, which means users need to know that they are writing asynchronous programs. - * Thanks to the convenience brought by ``std::function`` and the automatic memory recycling mechanism, we have delicately designed **the simply possible usage of asynchrony** for users. - * **No** user-mode threads concepts. On the one hand, performance is considered. On the other hand, we have the concept of computing tasks (threaded tasks) scheduling. - * In our design, **computing** is one kind of **asynchronous task**, which has no differences from communication. - * Computing tasks are scheduled by **independent thread groups** according to specific algorithms, please note that they **may not** be executed **immediately**. - * As we have such computing tasks, user-mode threads become meaningless, and therefore users must understand asynchrony. - * Because of the full asynchrony, almost all core calls are **short** and **non-blocking** operations. - * That’s why we **don’t** recommend users to **block** their programs in callback or do some complex calculations. However, it acceptable if the logic is quite simple. - * Brief summary of the usage: - * The user builds the program just like building a **series-parallel** circuit. The circuit can be generated **at the beginning** or **dynamically during the program running**. - * **We provide various electronic components** for users. For instance, one http request, one GPU matrix multiplication, and one parallel sorting can all be understood as a electronic component. - * Every electronic component has its **standard input and output**. At the meantime, every electronic component can be a **complicated circuit**, which has no necessary to be perceived by the users. - * For example, an http request may go through **multiple asynchronous processes** such as DNS, redirect, and retry, but the entire processes is just a **component** in the perspective of the users. - * Users can easily **define their own** components, including algorithms and some kind of communication. - * To implement **stateless protocols** is extremely simple. It may be a little bit complicated when the protocol includes login, library selection, etc., at this time, you can refer to the redis implementation. - * Through the powerful Upstream system, complex **service governance** can be realized, such as communication node selection, load balancing, circuit breaker and recovery, main and backup, etc. - * **In conclusion, this is an enterprise-level, elegantly designed asynchronous framework which can cover almost all high-performance back-end service requirements.** - -#### Tutorials： - * Client - * [Create your first task：wget](docs/tutorial-01-wget.md) - * [Implement redis set and get：redis_cli](docs/tutorial-02-redis_cli.md) - * [More features about series：wget_to_redis](docs/tutorial-03-wget_to_redis.md) - * Server - * [first server：http_echo_server](docs/tutorial-04-http_echo_server.md) - * [asynchronous server：http_proxy](docs/tutorial-05-http_proxy.md) - * Paralle task and Series　 - * [A simple parallel wget：parallel_wget](docs/tutorial-06-parallel_wget.md) - * Important topic - * [About error](docs/about-error.md) - * [About timeout](docs/about-timeout.md) - * [About DNS](docs/about-dns.md) - * [About exit](docs/about-exit.md) - * Computing tasks - * [Using the build-in algorithm factory：sort_task](docs/tutorial-07-sort_task.md) - * [User-define computing task：matrix_multiply](docs/tutorial-08-matrix_multiply.md) - * File asynchronous IO tasks - * [Http server with file IO：http_file_server](docs/tutorial-09-http_file_server.md) - * User-defined protocol basic usage - * [A simple user-defined portocol client/server](docs/tutorial-10-user_defined_protocol.md) - * Timing tasks and counting tasks - * [About timer](docs/about-timer.md) - * [About counter](docs/about-counter.md) - * Service governance - * [About service governance](docs/about-service-management.md) - * [More documents about upstream](docs/about-upstream.md) - * Connection context - * [About connection context](docs/about-connection-context.md) - * Built-in protocols - * [Asynchronous MySQL client：mysql_cli](docs/tutorial-12-mysql_cli.md) - -## Authors - -* **Xie Han** - *[xiehan@sogou-inc.com](mailto:xiehan@sogou-inc.com)* -* **Wu Jiaxu** - *[wujiaxu@sogou-inc.com](mailto:wujiaxu@sogou-inc.com)* -* **Li Yingxin** - *[liyingxin@sogou-inc.com](mailto:liyingxin@sogou-inc.com)* - -