Merge pull request #5 from sogou/master

merge from sogou git

CODE_OF_CONDUCT.md

@@ -0,0 +1,46 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others’ private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies within all project spaces, and it also applies when an individual is representing the project or its community in public spaces. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at xiehan@sogou-inc.com. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project’s leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

README.md

@@ -1,75 +1,50 @@
 [English version](README_en.md)
 
+## Sogou C++ Workflow
 [![license MIT](https://img.shields.io/badge/License-Apache-yellow.svg)](https://git.sogou-inc.com/wujiaxu/Filter/blob/master/LICENSE)
 [![C++](https://img.shields.io/badge/language-c++-red.svg)](https://en.cppreference.com/)
 [![platform](https://img.shields.io/badge/platform-linux%20%7C%20macos-lightgrey.svg)](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9)
 
-# Sogou C++ Workflow
-#### 搜狗公司的后端C++编程标准，是一套企业级的程序引擎。主要功能和特点：
-  * 这是一个基于C++11 ``std::function`` 的异步引擎。用于解决一切关于串行，并行和异步的问题。
-  * 作为网络框架，完全协议无关，并且直接面向应用。
-    * 可以当作一个异步redis客户端，也可以快速搭建一个Http服务器。
-    * 自定义协议非常方便，你可以快速的建立一个自己的RPC系统。
-      * Sogou RPC就是以它为基础开发，作为独立项目开源。该项目支持srpc，brpc和thrift等协议（[benchmark](https://github.com/holmes1412/sogou-rpc-benchmark)）。
-    * 支持SSL(依赖openssl)。支持TCP, UDP，SCTP等常用传输层协议。可支持SCTP上的SSL。但不支持UDP server。
-  * 原生包含了多种常有互联网协议的实现，并且以统一的方式使用。
-    * 目前支持http，redis，mysql和kafka协议，可以直接访问这些资源或搭建这些协议的server。
-    * 可能是目前市面上唯一C++全功能的mysql异步客户端。
-    * DNS协议开发中，目前使用系统库访问dns。
-  * 包含强大的计算任务调度功能。
-    * 计算任务可以和通信任务一样都可以加入任务流，有各自的调度器实现调度。
-    * 你可以不用网络功能，把它当并行编程工具来使用。
-    * 我们最大的目的就是在计算和通信非常复杂的情况下，最大程度的发挥节点的性能。
-    * 提供了一些常用算法实现，例如并行排序，MapReduce。
-    * 实际上，一切异步过程（如磁盘IO，GPU任务，定时器等）都可以被协同调度。
-      * 如果用户使用Linux系统，磁盘IO任务是是通过Linux底层aio实现，效率极高。
-  * 支持任何DAG结构的任务流。但大多数情况下，用户只需要使用串并联结构。
-  * 自带负载均衡和强大的服务治理等功能。
-  * 可以方便的与其他异步引擎协同使用。
-  * 支持streaming的通讯引擎开发中。
-  * 作为server使用时，支持多进程模式，支持精确的优雅重启。
+搜狗公司C++服务器引擎，支撑搜狗几乎所有后端C++在线服务，包括所有搜索服务，云输入法，在线广告等，每日处理超百亿请求。这是一个设计轻盈优雅的企业级程序引擎，可以满足大多数C++后端开发需求。  
+#### 你可以用来：
+* 快速搭建http服务器：
+~~~cpp
+#include <stdio.h>
+#include "workflow/WFHttpServer.h"
 
-#### 相关的技术要点：
-  * 目前项目支持Linux，macOS，FreeBSD，Windows等系统。需要安装cmake。
-    * Windows版暂时以独立[branch](https://github.com/sogou/workflow/tree/windows)发布，以iocp为异步通讯基础，对外接口一致。
-  * 该项目使用C/C++编写，使用者需要能比较熟练的使用C++编程。不依赖boost或asio，编译速度极快。
-  * 用到少量的C++11特征，用户需要会使用 ``std::function`` 和 ``std::move()`` 。
-  * 理论支持一切CPU架构，可以在32位或64位arm处理器上编译运行。Big endian CPU未测试。
-  * 项目需要依赖openssl。如果用户对SSL性能要求高，强烈建议使用openssl 1.1以上版本。
-  * 项目无其他依赖，但包含了snappy，lz4等几个压缩库的无改动源代码（kafka协议需要）。
+int main()
+{
+    WFHttpServer server([](WFHttpTask *task) {
+        task->get_resp()->append_output_body("<html>Hello World!</html>");
+    });
 
-#### 项目的一些设计特点：
-  * 项目基础用法非常简单，极易上手。而我们的一些设计也大幅降低了一般C++项目的使用难度。
-    * 尽量避免让用户派生，一切用户行为都用 ``std::function`` 来包装，例如：
-      * 任何任务结束后的callback。
-      * 计算任务里的算法。
-      * 一个server，也对应一个 ``std::function`` 。
-    * 尽量避免复杂的内存管理，一切任务和框架由工厂类产生，并自动内存回收。
-      * 任何任务在callback之后被自动delete。
-      * 任务中的数据（例如一个网络回复包，一个算法的结果），如果用户要保留，需要用 ``std::move()`` 把它移走。
-      * 内存回收是使用一套严密又符合自然逻辑的机制，我们不使用任何share_ptr。
-    * 尽量避免复杂的参数配置。
-      * 我们可配置的参数非常多，但您可以在对参数完全无感的情况下使用我们的系统。
-      * 如果您对程序行为，资源配比有特定需求，一定能找到对应配置项，以便发挥程序最高性能。
-  * 项目采用全异步的设计，并且对用户不透明，用户需要知道自己在写异步程序。
-    * 我们通过精心的设计的用户尽量简单的使用异步，得益于 ``std::function`` 带来的便利性，以及自动内存回收机制。
-    * 没有采用用户态线程的方式，一方面是性能考虑，另一方面原因是我们有计算任务（线程任务）调度的概念。
-      * 在我们的设计里，计算是一种异步任务，和通信并没有什么区别。
-      * 计算任务由独立的线程组依照特定算法进行调度，并不一定实时被执行。
-      * 因为有这种计算任务的存在，用户态线程似乎就没什么意义了，用户必然需要了解异步。
-    * 因为全异步的原因，我们几乎所有的核心调用都是短操作，不会阻塞。
-      * 我们也不鼓励用户在callback里阻塞程序，或做复杂的计算。但如果业务逻辑简单，这么做也无妨。
-  * 项目使用方法的简单总结：
-    * 用户如同搭建串并联电路一样搭建程序。电路可以一开始就生成好，也可以运行中动态产生。
-    * 我们提供各种电子元件供用户使用，一次http请求，一次GPU矩阵乘法，一次并行排序，都可以理解成为一个元件。
-    * 每个电子元件都有标准的输入输出。但每个电子元件内部，可能又是一个复杂电路，但用户并不感知。
-      * 比如一个http请求，可能要经历DNS，redirect，retry等多次异步过程，但对用户来讲，只是一个元件。
-    * 用户可以方便的定义自己的元件。比如一个算法，一次通信。
-      * 无状态的协议实现起来极其简单。如果协议有登录，选库等过程，则会复杂一些，可参考redis实现。
-    * 通过强大的Upstream系统，能实现非常复杂的服务治理，例如通信节点选择，负载均衡，熔断与恢复，主备等。
-    * 总之这是一个企业级的，设计优雅的异步编程框架，几乎可以覆盖一切高性能的后端服务需求。
+    if (server.start(8888) == 0) {  // start server on port 8888
+        getchar(); // press "Enter" to end.
+        server.stop();
+    }
 
-#### 使用教程：
+    return 0;
+}
+~~~
+* 作为万能异步客户端。目前支持``http``，``redis``，``mysql``和``kafka``协议。
+* 实现自定义协议client/server，构建自己的RPC系统。
+  * 搜狗RPC就是以它为基础，作为独立项目开源。支持``srpc``，``brpc``和``thrift``等协议（[benchmark](https://github.com/holmes1412/sogou-rpc-benchmark)）。
+* 构建异步任务流，支持常用的串并联，也支持更加复杂的DAG结构。
+* 作为并行编程工具使用。除了网络任务，我们也包含计算任务的调度。所有类型的任务都可以放入同一个流中。
+* 在``Linux``系统下作为文件异步IO工具使用，性能超过任何标准调用。磁盘IO也是一种任务。
+* 实现任何计算与通讯关系非常复杂的高性能高并发的后端服务。
+* 构建服务网格（service mesh）系统。
+  * 项目内置服务治理与负载均衡等功能。
+
+#### 编译和运行环境
+* 项目支持``Linux``，``macOS``，``Windows``等操作系统。
+  *  ``Windows``版暂时以独立branch发布，使用``iocp``实现异步网络。用户接口与``Linux``版一致。
+* 支持所有CPU平台，包括32或64位``x86``处理器，大端或小端``arm``处理器。
+* 需要依赖于``OpenSSL``，推荐``OpenSSL 1.1``及以上版本。
+* 项目使用了``C++11``标准，需要用支持``C++11``的编译器编译。但不依赖``boost``或``asio``。
+* 项目无其它依赖，但是包含了``lz4``，``zstd``和``snappy``几个压缩库的无改动源代码（``kafka``协议需要）。
+
+# 试一下！
   * Client基础
     * [创建第一个任务：wget](docs/tutorial-01-wget.md)
     * [实现一次redis写入与读出：redis_cli](docs/tutorial-02-redis_cli.md)
@@ -82,6 +57,7 @@
   * 几个重要的话题
     * [关于错误处理](docs/about-error.md)
     * [关于超时](docs/about-timeout.md)
+	* [关于全局配置](docs/about-config.md)
     * [关于DNS](docs/about-dns.md)
     * [关于程序退出](docs/about-exit.md)
   * 计算任务
@@ -102,10 +78,52 @@
   * 内置协议用法
     * [异步MySQL客户端：mysql_cli](docs/tutorial-12-mysql_cli.md)
 
-## Authors
+#### 系统设计特点
+
+我们认为，一个典型的后端程序由以下三个部分组成，并且完全独立开发。
+* 协议
+  * 大多数情况下，用户使用的是内置的通用网络协议，例如http，redis或各种rpc。
+  * 用户可以方便的自定义网络协议，只需提供序列化和反序列化函数，就可以定义出自己的client/server。
+* 算法
+  * 在我们的设计里，算法是与协议对称的概念。
+    * 如果说协议的调用是rpc，算法的调用就是一次apc（Async Procedure Call）。
+  * 我们提供了一些通用算法，例如sort，merge，psort，reduce，可以直接使用。
+  * 与自定义协议相比，自定义算法的使用要常见得多。任何一次边界清晰的复杂计算，都应该包装成算法。
+* 任务流
+  * 任务流就是实际的业务逻辑，就是把开发好的协议与算法放在流程图里使用起来。
+  * 典型的任务流是一个闭合的串并联图。复杂的业务逻辑，可能是一个非闭合的DAG。
+  * 任务流图可以直接构建，也可以根据每一步的结果动态生成。所有任务都是异步执行的。
+
+基础任务，任务工厂与复合任务
+* 我们系统中包含六种基础任务：通讯，文件IO，CPU，GPU，定时器，计数器。
+* 一切任务都由任务工厂产生，并且在callback之后自动回收。
+  * server任务是一种特殊的通讯任务，由框架调用任务工厂产生，通过process函数交给用户。
+* 大多数情况下，用户通过任务工厂产生的任务，都是一个复合任务，但用户并不感知。
+  * 例如，一次http请求，可能包含许多次异步过程（DNS，重定向），但对用户来讲，就是一次通信任务。
+  * 文件排序，看起来就是一个算法，但其实包括复杂的文件IO与CPU计算的交互过程。
+  * 如果把业务逻辑想象成用设计好的电子元件搭建电路，那么每个电子元件内部可能又是一个复杂电路。
+
+异步性和基于``C++11 std::function``的封装
+* 不是基于用户态协程。使用者需要知道自己在写异步程序。
+* 一切调用都是异步执行，几乎不存在占着线程等待的操作。
+  * 虽然我们也提供一些便利的半同步接口，但并不是核心的功能。
+* 尽量避免派生，以``std::function``封装用户行为，包括：
+  * 任何任务的callback。
+  * 任何server的process。符合``FaaS``（Function as a Service）思想。
+  * 一个算法的实现，简单来讲也是一个``std::function``。但算法也可以用派生实现。
+
+内存回收机制
+* 任何任务都会在callback之后被自动内存回收。如果创建的任务不想运行，则需要通过dismiss方法释放。
+* 任务中的数据，例如网络请求的resp，也会随着任务被回收。此时用户可通过``std::move()``把需要的数据移走。
+* SeriesWork和ParallelWork是两种框架对象，同样在callback之后被回收。
+* 项目中不使用``std::shared_ptr``来管理内存。
+
+#### 更多设计文档
+持续更新中……
+
+
+#### Authors
 
 * **Xie Han** - *[xiehan@sogou-inc.com](mailto:xiehan@sogou-inc.com)*
 * **Wu Jiaxu** - *[wujiaxu@sogou-inc.com](mailto:wujiaxu@sogou-inc.com)*
 * **Li Yingxin** - *[liyingxin@sogou-inc.com](mailto:liyingxin@sogou-inc.com)*
-
-

README_en.md

@@ -1,76 +1,52 @@
 [中文版](README.md)
 
+## Sogou C++ Workflow
 [![license MIT](https://img.shields.io/badge/License-Apache-yellow.svg)](https://git.sogou-inc.com/wujiaxu/Filter/blob/master/LICENSE)
 [![C++](https://img.shields.io/badge/language-c++-red.svg)](https://en.cppreference.com/)
 [![platform](https://img.shields.io/badge/platform-linux%20%7C%20macos-lightgrey.svg)](#%E9%A1%B9%E7%9B%AE%E7%9A%84%E4%B8%80%E4%BA%9B%E8%AE%BE%E8%AE%A1%E7%89%B9%E7%82%B9)
 
-# Sogou C++ Workflow
-#### As the backend C++ programming standard in Sogou, Workflow is an industrial-grade programming engine. 
-#### Main functions and features:
-  * An **asynchronous engine** based on **C++11** ``std::function`` which aims to solve all the **serial, parallel and asynchronous** problems.
-  * As a network framework, it is completely **protocol-agnostic** and directly facing applications.
-    * It can either be used as a Redis **client** or an Http **server**.
-    * Convenient to **customize protocols**, so you can quickly build your own RPC systems.
-      * Sogou RPC is developed based on Sogou Workflow and is open source as an independent project. The project supports srpc, brpc and thrift protocols ([benchmark](https://github.com/holmes1412/sogou-rpc-benchmark)).
-    * Support **SSL** (depends on openssl). Support **TCP, UDP, SCTP** and other common transport layer protocols. Support SSL on **SCTP**. Not support UDP server.
-  * Natively contains a variety of **common Internet protocol** implementations which are used in a unified way.
-    * Currently support **http, redis, mysql** and **kafka** protocols. You can directly access these resources or build **servers** for these protocols.
-    * Highly likely the only C++ full-featured **mysql asynchronous client** on the market.
-    * **DNS** protocol is being developing and currently we use the system library to access DNS.
-  * Powerful feature for **scheduling computing tasks**
-    * Computing task, as well as communication task, can be added into the task flow and they’re scheduled separately by their corresponding scheduler.
-    * You can use it as a parallel programming engine **without** the network features.
-    * Our biggest goal is to **maximize the performance** of every node when the calculation and communication environment is very complex.
-    * Some **common algorithm** implementations are provided, such as parallel sorting and MapReduce.
-    * In fact, all asynchronous processes (such as disk IO, GPU tasks, timers, etc.) **can be scheduled in coordination**.
-      * On the Linux system, the disk IO task is realized through the Linux underlying aio, which is extremely efficient.
-  * Support any task flow with **DAG** structure. However, in most cases, users only need **series-parallel** structure.
-  * Built-in **load balancing** and powerful **service governance** features.
-  * Easily used in conjunction with other asynchronous engines.
-  * **Streaming** communication engine is being developed.
-  * When working as a server, it supports **multi-processes** mode and supports precise **graceful restart**.
+As **Sogou`s C++ server engine**, workflow supports almost all **back-end C++ online services** of Sogou, including all search services, cloud input method，online advertisements, etc., handling more than **10 billion** requests every day. This is an **enterprise-level programming engine** with light and elegantly designed which can satisfy most C++ back-end development requirements.
 
-#### Building
-  * Support **Linux, macOS, FreeBSD, Windows** and other systems so far. Installing **cmake** is necessary.
-    * Windows version is temporarily released as an independent [branch](https://github.com/sogou/workflow/tree/windows), which uses **iocp** as the basis for asynchronous communication and mean while, keeping the **same external interface**.
-  * As written in C/C++, it requires the users being able to proficiently use C++ programing. It **does not** rely on boost or asio, therefor the compiling speed is extremely fast.
-  * It contains a few C++11 features, so users should be able to use ``std::function`` and ``std::move``.
-  * Theoretically support all CPU architectures and can be compiled and run on **32-bit** or **64-bit arm processors**. Big endian CPU is not tested.
-  * **OpenSSL** is required. If users expect high performance of SSL, OpenSSL 1.1 or higher is strongly recommended.
-  * **No other dependencies**. Several compression libraries such as snappy and lz4  is contained by their unmodified source (required by the Kafka protocol).
+#### You can use it:
+* To quickly build an **Http server**:
+~~~cpp
+#include <stdio.h>
+#include "workflow/WFHttpServer.h"
 
-#### Some design features
-  * The basic usage is very simple and handy. Some features are designed to greatly reduce the difficulty of programming with general C++ projects.
-    * To **avoid** users to **derive** as much as possible, all user behaviors are wrapped with std::function, for example:
-      * the **callback** after every task ends
-      * the **algorithms** in computing tasks
-      * one server corresponds to one ``std::function``
-    * Trying to avoid complicated memory management, all tasks and frameworks are generated by **factory** classes, and their memory is **recycled automatically**. Which means,
-      * Every task is **automatically deleted after its callback**.
-      * If the users want to keep any data in the task (such as a network reply packet or the result of an algorithm), they need to use ``std::move`` to move it.
-      * We treat memory recycle as a strict and naturally logical mechanism, so we **don’t** use share_ptr.
-    * Avoid using complicated parameter configuration.
-      * Actually we have a lot of **configurable parameters**, though you can use our system **without** feeling the existence of parameters.
-      * If you have specific requirements for program behavior and resource ratio, you can definitely find the corresponding parameter configuration items in order to maximize the performance of you program.
-  * The project adopts a fully asynchronous design and is not transparent to users, which means users need to know that they are writing asynchronous programs.
-    * Thanks to the convenience brought by ``std::function`` and the automatic memory recycling mechanism, we have delicately designed **the simply possible usage of asynchrony** for users.
-    * **No** user-mode threads concepts. On the one hand, performance is considered. On the other hand, we have the concept of computing tasks (threaded tasks) scheduling.
-      * In our design, **computing** is one kind of **asynchronous task**, which has no differences from communication.
-      * Computing tasks are scheduled by **independent thread groups** according to specific algorithms, please note that they **may not** be executed **immediately**.
-      * As we have such computing tasks, user-mode threads become meaningless, and therefore users must understand asynchrony.
-    * Because of the full asynchrony, almost all core calls are **short** and **non-blocking** operations.
-      * That’s why we **don’t** recommend users to **block** their programs in callback or do some complex calculations. However, it acceptable if the logic is quite simple.
-   * Brief summary of the usage:
-    * The users can build the program just like building a **series-parallel** circuit. The circuit can be generated **at the beginning** or **dynamically generated during the program running**.
-    * **We provide various electronic components** for users. For instance, one http request, one GPU matrix multiplication, and one parallel sorting can all be understood as a electronic component.
-    * Every electronic component has its **standard input and output**. At the meantime, every electronic component can be a **complicated circuit**, which has no necessary to be perceived by the users.
-    * For example, an http request may go through **multiple asynchronous processes** such as DNS, redirect, and retry, but the entire processes is just a **component** in the perspective of the users.
-    * Users can easily **define their own** components, including algorithms and some kind of communication.
-      * To implement **stateless protocols** is extremely simple. It may be a little bit complicated when the protocol includes login, library selection, etc., at this time, you can refer to the redis implementation.
-    * Through the powerful Upstream system, complex **service governance** can be realized, such as communication node selection, load balancing, circuit breaker and recovery, master and slave, etc.
-    * **In conclusion, this is an enterprise-level, elegantly designed asynchronous framework which can cover almost all high-performance back-end service requirements.**
+int main()
+{
+    WFHttpServer server([](WFHttpTask *task) {
+        task->get_resp()->append_output_body("<html>Hello World!</html>");
+    });
 
-#### Tutorials：
+    if (server.start(8888) == 0) {  // start server on port 8888
+        getchar(); // press "Enter" to end.
+        server.stop();
+    }
+
+    return 0;
+}
+~~~
+* As a **powerful asynchronous client**. Currently supports ``http``, ``redis``, ``mysql`` and ``kafka`` protocols.
+* To realize **user-defined protocol client/server** and build your own **RPC system**.
+  * Sogou RPC is based on it and open source as an independent project, which supports srpc, brpc and thrift protocol ([benchmark](https://github.com/holmes1412/sogou-rpc-benchmark)).
+* To build **asynchronous task flow**, support common **series** and **parallel** structures, and also support more complex **DAG** structures.
+* As a **parallel programming tool**. In addition to **network tasks**, we also include **the scheduling of computing tasks**. All types of tasks can be put into **the same** task flow.
+* As a **file asynchronous IO tool** under ``Linux`` system, with a high performance exceeding any system call. Disk IO is also a task.
+* To realize any **high-performance** and **high-concurrency** back-end service with a very complex relationship between computing and communication.
+* To build a **service mesh** system.
+  * The project has built-in **service governance** and **load balancing** features.
+
+#### Compile and run environment
+
+* This project supports ``Linux``, ``macOS``, ``Windows`` and other operating systems.
+  * ``Windows`` version is temporarily released as an independent branch, using ``iocp`` to implement asynchronous networking. All user interfaces are consistent with the ``Linux`` version.
+* Supports all CPU platforms, including 32 or 64-bit ``x86`` processors, big-endian or little-endian ``arm`` processors.
+* Relies on ``OpenSSL``, recommending ``OpenSSL 1.1`` and above.
+* Uses the ``C++11`` standard and therefore, needs to be compiled with a compiler which supports ``C++11``. Does not rely on ``boost`` or ``asio``.
+* No other dependencies. However, it contains the unmodified source code of several compression libraries such as ``lz4``, ``zstd`` and ``snappy`` (required by the ``Kafka`` protocol).
+
+# Try it!
   * Client
     * [Create your first task：wget](docs/tutorial-01-wget.md)
     * [Implement redis set and get：redis_cli](docs/tutorial-02-redis_cli.md)
@@ -83,6 +59,7 @@
   * Important topics
     * [About error](docs/about-error.md)
     * [About timeout](docs/about-timeout.md)
+    * [About global configuration](docs/about-config.md)
     * [About DNS](docs/about-dns.md)
     * [About exit](docs/about-exit.md)
   * Computing tasks
@@ -103,10 +80,52 @@
   * Built-in protocols
     * [Asynchronous MySQL client：mysql_cli](docs/tutorial-12-mysql_cli.md)
 
+#### System design features
+
+We believe that a typical back-end program consists of the following three parts and should be developed completely independently.
+* Protocol
+  * In most cases, users use built-in common network protocols, such as http, redis or various rpc.
+  * Users can also easily customize user-defined network protocol,  at the mean time they only need to provide serialization and deserialization functions to define their own client/server.
+* Algorithm
+  * In our design, algorithm is a symmetrical concept with protocol.
+    * If protocol call is rpc, then algorithm call is an apc (Async Procedure Call).
+  * We have provided some general algorithms, such as sort, merge, psort, reduce, which can be used directly.
+  * Compared with user-defined protocol, user-defined algorithm is much more common. Any complex calculation with clear boundaries should be packaged into an algorithm.
+* Task flow
+  * Task flow is the actual bussiness logic, which is to put the protocols and algorithms into the flow graph for use.
+  * The typical task flow is a closed series-parallel graph. Complex business logic may be a non-closed DAG.
+  * The task flow graph can be constructed directly or dynamically generated based on the results of each step. All tasks are executed asynchronously.
+
+Basic task, task factory and complex task
+* Our system contains six basic tasks: communication, file IO, CPU, GPU, timer, and counter.
+* All tasks are generated by the task factory and automatically recycled after callback.
+  * Server task is one kind of special communication task, generated by the framework which calls the task factory, and handed over to the user through the process function.
+  * In most cases, the task generated by the user through the task factory is a complex task, which has no necessary to be perceived by the user.
+  * For example, an Http request may include many asynchronous processes (DNS, redirection), but for the user, it is just a communication task.
+  * File sorting seems to be an algorithm, but it actually includes many complex interaction processes between file IO and CPU calculation.
+  * If you think of business logic as building circuits with well-designed electronic components, then each electronic component may be a complex circuit.
+
+Asynchrony and encapsulation based on ``C++11 std::function``
+
+* Not based on user mode coroutines. Users need to know that they are writing asynchronous programs.
+* All calls are executed asynchronously, and there are almost no operations to wait for threads.
+  * Although we also provide some convenient semi-synchronous interfaces, they are not core features.
+* Please avoid derivation.Try to encapsulate user behavior with ``std::function`` instead, including:
+  * The callback of any task.
+  * Any server process. This conforms to the ``FaaS`` (Function as a Service) idea.
+  * The realization of an algorithm is simply a ``std::function``. But the algorithm can also be implemented by derivative.
+
+Memory reclamation mechanism
+* Every task will be automatically reclaimed after the callback. If a task is created but does not want to run, the user needs to release it through the dismiss method.
+* Any data in the task, such as the response of the network request, will also be recycled with the task. At this time, the user can use ``std::move()`` to move the required data.
+* SeriesWork and ParallelWork are two kinds of framework objects, which are also recycled after their callback.
+* This project doesn’t use ``std::shared_ptr`` to manage memory.
+
+#### More design documents
+To be continued...
+
 ## Authors
 
 * **Xie Han** - *[xiehan@sogou-inc.com](mailto:xiehan@sogou-inc.com)*
 * **Wu Jiaxu** - *[wujiaxu@sogou-inc.com](mailto:wujiaxu@sogou-inc.com)*
 * **Li Yingxin** - *[liyingxin@sogou-inc.com](mailto:liyingxin@sogou-inc.com)*
-
-

benchmark/CMakeLists.txt

@@ -0,0 +1,41 @@
+cmake_minimum_required(VERSION 3.6)
+
+set(CMAKE_BUILD_TYPE RelWithDebInfo CACHE STRING "build type")
+
+project(benchmark
+		LANGUAGES C CXX
+)
+
+set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${PROJECT_SOURCE_DIR})
+
+find_library(LIBRT rt)
+find_package(OpenSSL REQUIRED)
+find_package(workflow REQUIRED CONFIG HINTS ..)
+include_directories(${OPENSSL_INCLUDE_DIR} ${WORKFLOW_INCLUDE_DIR})
+link_directories(${WORKFLOW_LIB_DIR})
+
+if (WIN32)
+		set(CMAKE_C_FLAGS   "${CMAKE_C_FLAGS}   /MP /wd4200")
+		set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /MP /wd4200 /std:c++14")
+else ()
+		set(CMAKE_C_FLAGS   "${CMAKE_C_FLAGS}   -Wall -fPIC -pipe -std=gnu90")
+		set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall -fPIC -pipe -std=c++11 -fno-exceptions")
+endif ()
+
+set(BENCHMARK_LIST
+	benchmark-01-http_server
+	benchmark-02-http_server_long_req
+)
+
+if (APPLE)
+	set(WORKFLOW_LIB workflow pthread z OpenSSL::SSL OpenSSL::Crypto)
+else ()
+	set(WORKFLOW_LIB workflow ${LIBRT})
+endif ()
+
+foreach(src ${BENCHMARK_LIST})
+	string(REPLACE "-" ";" arr ${src})
+	list(GET arr -1 bin_name)
+	add_executable(${bin_name} ${src}.cc)
+	target_link_libraries(${bin_name} ${WORKFLOW_LIB})
+endforeach()

benchmark/GNUmakefile

@@ -0,0 +1,27 @@
+ROOT_DIR := $(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))
+ALL_TARGETS := all clean
+MAKE_FILE := Makefile
+
+DEFAULT_BUILD_DIR := build
+BUILD_DIR := $(shell if [ -f $(MAKE_FILE) ]; then echo "."; else echo $(DEFAULT_BUILD_DIR); fi)
+CMAKE3 := $(shell if which cmake3>/dev/null ; then echo cmake3; else echo cmake; fi;)
+
+.PHONY: $(ALL_TARGETS)
+
+all:
+	mkdir -p $(BUILD_DIR)
+ifeq ($(DEBUG),y)
+	cd $(BUILD_DIR) && $(CMAKE3) -D CMAKE_BUILD_TYPE=Debug $(ROOT_DIR)
+else
+	cd $(BUILD_DIR) && $(CMAKE3) $(ROOT_DIR)
+endif
+	make -C $(BUILD_DIR) -f Makefile
+
+clean:
+ifeq ($(MAKE_FILE), $(wildcard $(MAKE_FILE)))
+	-make -f Makefile clean
+else ifeq (build, $(wildcard build))
+	-make -C build clean
+endif
+	rm -rf build
+

benchmark/README.md

@@ -0,0 +1,238 @@
+# 性能测试
+
+Sogou C++ Workflow是一款性能优异的网络框架，本文介绍我们进行的性能测试，
+包括方案、代码、结果，以及与其他同类产品的对比。
+
+更多场景下的实验正在进行中，本文将持续更新。
+
+## HTTP Server
+
+HTTP Client/Server是Sogou C++ Workflow常见的应用场景，
+我们首先对Server端进行实验。
+
+### 环境
+
+我们部署了两台相同机器作为Server和Client，软硬件配置如下：
+
+| 软硬件 | 配置 |
+|:---:|:---|
+| CPU | 40 Cores, x86_64, Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz |
+| Memory | 192GB |
+| NIC | 25000Mbps |
+| OS | CentOS 7.8.2003 |
+| Kernel | Linux version 3.10.0-1127.el7.x86_64 |
+| GCC | 4.8.5 |
+
+两者间`ping`测得的RTT为0.1ms左右。
+
+### 对照组
+
+我们选择nginx和brpc作为对照组。
+选择前者是因为它在生产中部署十分广泛，性能不俗；
+对于后者，我们在本次实验中只关注HTTP Server方面的能力，
+其他的特性已有[单独的实验][Sogou RPC Benchmark]进行更为详尽的测试。
+
+事实上，我们也对此二者之外的其他某些框架同时进行了实验，
+但结果其性能表现相差较远，因此未在本文中体现。
+
+后续我们将选取更多合适的框架加入对比测试中。
+
+### Client工具
+
+本次实验我们使用的压测工具为[wrk][wrk]和[wrk2][wrk2]。
+前者适合测试特定并发下的QPS极限和延时，
+后者适合在特定QPS下测试延时分布。
+
+我们也尝试过使用其他测试工具，例如[ab][ab]等，但无法打出足够的压力。
+有鉴于此，我们也在着手开发基于Sogou C++ Workflow的benchmark工具。
+
+### 变量和指标
+
+一般而言，对网络框架的性能测试，切入的角度可谓纷繁多样。
+通过控制不同的变量、观测不同的指标，可以探究程序在不同场景下的适应能力。
+
+本次实验，我们选择其中最普遍常见的变量和指标：
+通过控制Client并发度和承载数据的大小，来测试QPS和延时的变化情况。
+另外，我们还测试了在掺杂慢请求的正常请求的延时分布。
+
+下面依次介绍两个测试场景。
+
+### 测试方法
+
+#### 启动http server
+1. 编译benchmark
+2. 进入到benchmark目录，执行 
+
+```
+./http_server 12 9000 11
+```
+
+说明: 启动参数分别为线程数、端口和响应的随机字符串长度。
+
+### wrk测试
+
+```
+wrk --latency -d10 -c200 --timeout 8 -t 6 http://127.0.0.1:9000
+```
+**命令行解释**
+
+-c200: 启动200个连接
+
+-t6: 开启6个线程做压力测试
+
+-d10: 压测持续10s
+
+--timeout 8: 连接超时时间8s
+
+### 不同并发度和数据长度下的QPS和延时
+
+#### 代码和配置
+
+我们搭建了一个极其简约的HTTP服务器，
+忽略掉所有的业务逻辑，
+将测试点聚焦在纯粹的网络框架性能上。
+
+代码片段如下，
+完整代码移步[这里][benchmark-01 Code]。
+
+```cpp
+// ...
+
+auto * resp = task->get_resp();
+resp->add_header_pair("Date", timestamp);
+resp->add_header_pair("Content-Type", "text/plain; charset=UTF-8");
+resp->append_output_body_nocopy(content.data(), content.size());
+
+// ...
+```
+
+可以从上述代码中看到，
+对于到来的任何HTTP请求，
+我们都会返回一段固定的内容作为Body，
+并设置必要的Header，
+包括代码中指明的`content-type`、`date`，
+以及自动填充的`connection`和`content-length`。
+
+HTTP Body的固定内容是在Server启动时随机生成的ASCII字符串，
+其长度可以通过启动参数配置。
+同时可以配置的还有使用的poller线程数和监听的端口号。
+前者我们在本次测试中固定为16，
+因此Sogou C++ Workflow将使用16个poller线程和20个handler线程（默认配置）。
+
+对于nginx和brpc，
+我们也构建了相同的返回内容，
+并为nginx配置了40个进程、
+brpc配置了40个线程。
+
+
+#### 变量
+
+我们控制并发度在`[1, 2K]`之间翻倍增长，
+数据长度在`[16B, 64KB]`之间翻倍增长，
+两者正交。
+
+#### 指标
+
+鉴于并发度和数据长度组合之后数量较多，
+我们选择其中部分数据绘制为曲线。
+
+##### 固定数据长度下QPS与并发度关系
+
+![Concurrency and QPS][Con-QPS]
+
+上图可以看出，当数据长度保持不变，
+QPS随着并发度提高而增大，后趋于平稳。
+此过程中Sogou C++ Workflow一直有明显优势，
+高于brpc和nginx。
+特别是数据长度为64和512的两条曲线，
+并发度足够的时候，可以保持500K的QPS。
+
+注意上图中nginx-64与nginx-512的曲线重叠度很高，
+不易辨识。
+
+##### 固定并发度下QPS与数据长度关系
+
+![Body Length and QPS][Len-QPS]
+
+上图可以看出，当并发度保持不变，
+随着数据长度的增长，
+QPS保持平稳至4K时下降。
+此过程中，Sogou C++ Workflow也一直保持优势。
+
+##### 固定数据长度下延时与并发度关系
+
+![Concurrency and Latency][Con-Lat]
+
+上图可以看出，保持数据长度不变，
+延时随并发度提高而有所上升。
+此过程中，Sogou C++ Workflow略好于brpc，
+大好于nginx。
+
+##### 固定并发度下延时与数据长度关系
+
+![Body Length and Latency][Len-Lat]
+
+上图可以看出，并发度保持不变时，
+增大数据长度，造成延时上升。
+此过程中，Sogou C++ Workflow好于nginx，
+好于brpc。
+
+### 掺杂慢请求的延时分布
+
+#### 代码
+
+我们在上一个测试的基础上，简单添加了一个慢请求的逻辑，
+模拟业务场景中可能出现的特殊情况。
+
+代码片段如下，
+完整代码请移步[这里][benchmark-02 Code]。
+
+```cpp
+// ...
+
+if (std::strcmp(uri, "/long_req/") == 0)
+{
+    auto timer_task = WFTaskFactory::create_timer_task(microseconds, nullptr);
+    series_of(task)->push_back(timer_task);
+}
+// ...
+```
+
+我们在Server的process里进行判断，
+如果访问的是特定的路径，
+则添加一个`WFTimerTask`到Series的末尾，
+能够模拟一个异步耗时处理过程。
+类似地，对brpc使用`bthread_usleep()`函数进行异步睡眠。
+
+#### 配置
+
+在本次实验中，我们固定并发度为1024，数据长度为1024字节，
+分别以QPS为20K、100K和200K进行正常请求测试，
+测绘延时；
+与此同时，有另一路压力，进行慢请求，
+QPS是上述QPS的1%，
+数据不计入统计。
+慢请求的时长固定为5ms。
+
+#### 延时CDF图
+
+![Latency CDF][Lat CDF]
+
+从上图可以看出，当QPS为20K时，
+Sogou C++ Workflow略次于brpc；
+当QPS为100K时，两者几乎相当；
+当QPS为200K时，Sogou C++ Workflow略好于brpc。
+总之，可以认为两者在这方面旗鼓相当。
+
+
+[Sogou RPC Benchmark]: https://github.com/holmes1412/sogou-rpc-benchmark
+[wrk]: https://github.com/wg/wrk
+[wrk2]: https://github.com/giltene/wrk2
+[ab]: https://httpd.apache.org/docs/2.4/programs/ab.html
+[benchmark-01 Code]: benchmark-01-http_server.cc
+[benchmark-02 Code]: benchmark-02-http_server_long_req.cc
+[Con-QPS]: ../docs/img/benchmark-01.png
+[Len-QPS]: ../docs/img/benchmark-02.png
+[Con-Lat]: ../docs/img/benchmark-03.png
+[Len-Lat]: ../docs/img/benchmark-04.png
+[Lat CDF]: ../docs/img/benchmark-05.png

benchmark/benchmark-01-http_server.cc

@@ -0,0 +1,58 @@
+#include <csignal>
+
+#include <workflow/WFHttpServer.h>
+#include <workflow/WFGlobal.h>
+#include <workflow/WFFacilities.h>
+
+#include "util/args.h"
+#include "util/content.h"
+#include "util/date.h"
+
+static WFFacilities::WaitGroup wait_group{1};
+
+void signal_handler(int)
+{
+	wait_group.done();
+}
+
+int main(int argc, char ** argv)
+{
+	size_t pollers;
+	unsigned short port;
+	size_t length;
+
+	if (parse_args(argc, argv, pollers, port, length) != 3)
+	{
+		return -1;
+	}
+
+	std::signal(SIGINT, signal_handler);
+	std::signal(SIGTERM, signal_handler);
+
+	WFGlobalSettings settings = GLOBAL_SETTINGS_DEFAULT;
+	settings.poller_threads = pollers;
+	WORKFLOW_library_init(&settings);
+
+	const std::string content = make_content(length);
+	WFHttpServer server([&content](WFHttpTask * task)
+	{
+		auto * resp = task->get_resp();
+
+		char timestamp[32];
+		date(timestamp, sizeof(timestamp));
+		resp->add_header_pair("Date", timestamp);
+
+		resp->add_header_pair("Content-Type", "text/plain; charset=UTF-8");
+
+		resp->append_output_body_nocopy(content.data(), content.size());
+	});
+
+	if (server.start(port) == 0)
+	{
+		wait_group.wait();
+		server.stop();
+	}
+
+	return 0;
+}
+

benchmark/benchmark-02-http_server_long_req.cc

@@ -0,0 +1,68 @@
+#include <csignal>
+#include <cstring>
+
+#include <workflow/WFHttpServer.h>
+#include <workflow/WFGlobal.h>
+#include <workflow/WFFacilities.h>
+
+#include "util/args.h"
+#include "util/content.h"
+#include "util/date.h"
+
+static WFFacilities::WaitGroup wait_group{1};
+
+void signal_handler(int)
+{
+	wait_group.done();
+}
+
+int main(int argc, char ** argv)
+{
+	size_t pollers;
+	unsigned short port;
+	size_t length;
+	size_t microseconds;
+
+	if (parse_args(argc, argv, pollers, port, length, microseconds) != 4)
+	{
+		return -1;
+	}
+
+	std::signal(SIGINT, signal_handler);
+	std::signal(SIGTERM, signal_handler);
+
+	WFGlobalSettings settings = GLOBAL_SETTINGS_DEFAULT;
+	settings.poller_threads = pollers;
+	WORKFLOW_library_init(&settings);
+
+	const std::string content = make_content(length);
+	WFHttpServer server([&content, &microseconds](WFHttpTask * task)
+	{
+		auto resp = task->get_resp();
+
+		char timestamp[32];
+		date(timestamp, sizeof(timestamp));
+		resp->add_header_pair("Date", timestamp);
+
+		resp->add_header_pair("Content-Type", "text/plain; charset=UTF-8");
+
+		resp->append_output_body_nocopy(content.data(), content.size());
+
+		auto req = task->get_req();
+		auto uri = req->get_request_uri();
+		if (std::strcmp(uri, "/long_req/") == 0)
+		{
+			auto timer_task = WFTaskFactory::create_timer_task(microseconds, nullptr);
+			series_of(task)->push_back(timer_task);
+		}
+	});
+
+	if (server.start(port) == 0)
+	{
+		wait_group.wait();
+		server.stop();
+	}
+
+	return 0;
+}
+

benchmark/util/args.h

@@ -0,0 +1,95 @@
+#ifndef _BENCHMARK_ARGS_H_
+#define _BENCHMARK_ARGS_H_
+
+#include <algorithm>
+#include <numeric>
+#include <string>
+
+namespace details
+{
+	inline bool extract(const char * p, size_t & t)
+	{
+		char * e;
+		long long ll = std::strtoll(p, &e, 0);
+		if (*e || ll < 0)
+		{
+			return false;
+		}
+		t = static_cast<size_t>(ll);
+		return true;
+	}
+
+	inline bool extract(const char * p, unsigned short & t)
+	{
+		char * e;
+		long long ll = std::strtoll(p, &e, 0);
+		if (*e
+		    || ll < static_cast<long long>(std::numeric_limits<unsigned short>::min())
+		    || ll > static_cast<long long>(std::numeric_limits<unsigned short>::max())
+			)
+		{
+			return false;
+		}
+		t = static_cast<unsigned short>(ll);
+		return true;
+	}
+
+	inline bool extract(const char * p, std::string & t)
+	{
+		t = p;
+		return true;
+	}
+
+	inline bool extract(const char * p, const char *& t)
+	{
+		t = p;
+		return true;
+	}
+
+	template <typename ARG>
+	inline int parse_one(bool & flag, char **& p, char ** end, ARG & arg)
+	{
+		if (flag && (flag = p < end) && (flag = extract(*p, arg)))
+		{
+			p++;
+		}
+		return 0;
+	}
+
+	template <typename ... ARGS>
+	inline size_t parse_all(char ** begin, char ** end, ARGS & ... args)
+	{
+		bool flag = true;
+		char ** p = begin;
+		static_cast<void>(std::initializer_list<int>{parse_one(flag, p, end, args) ...});
+		return p - begin;
+	}
+
+	template <typename ... ARGS>
+	inline size_t parse_args(int & argc, char ** argv, ARGS & ... args)
+	{
+		if (argc <= 1)
+		{
+			return 0;
+		}
+
+		size_t length = argc - 1;
+		char ** begin = argv + 1;
+		char ** end = begin + length;
+
+		size_t done = parse_all(begin, end, args ...);
+		std::rotate(begin, begin + done, end);
+		std::reverse(end - done, end);
+
+		argc -= done;
+		return done;
+	}
+}
+
+template <typename ... ARGS>
+inline static size_t parse_args(int & argc, char ** argv, ARGS & ... args)
+{
+	return details::parse_args(argc, argv, args ...);
+}
+
+#endif //_BENCHMARK_ARGS_H_

benchmark/util/content.h

@@ -0,0 +1,21 @@
+#ifndef _BENCHMARK_CONTENT_H_
+#define _BENCHMARK_CONTENT_H_
+
+#include <random>
+#include <string>
+
+static inline std::string make_content(size_t length)
+{
+	std::mt19937_64 gen{42};
+	std::uniform_int_distribution<int> dis{32, 126};
+
+	std::string s;
+	s.reserve(length);
+	for (size_t i = 0; i < length; i++)
+	{
+		s.push_back(static_cast<char>(dis(gen)));
+	}
+	return s;
+}
+
+#endif //_BENCHMARK_CONTENT_H_

benchmark/util/date.h

@@ -0,0 +1,15 @@
+#ifndef _BENCHMARK_DATE_H_
+#define _BENCHMARK_DATE_H_
+
+#include <ctime>
+
+static inline void date(char * buf, size_t n)
+{
+	auto tt = std::time(nullptr);
+	std::tm cur{};
+	// gmtime_r(&tt, &cur);
+	localtime_r(&tt, &cur);
+	strftime(buf, n, "%a, %d %b %Y %H:%M:%S %Z", &cur);
+}
+
+#endif //_BENCHMARK_DATE_H_

docs/about-config.md

@@ -0,0 +1,77 @@
+# 关于全局配置
+
+全局配置用于配置全局默认参数，以适应的实际业务需求，提升程序性能。
+全局配置的修改必须在使用框架任何调用之前，否则修改可能无法生效。
+另外，一些全局配置选项，可以被upstream配置覆盖。这部分请参考upstream相关文档。
+
+# 修改默认配置
+
+在[WFGlobal.h](../src/manager/WFGlobal.h)里，包含了全局配置的结构体与默认值：
+~~~cpp
+struct WFGlobalSettings
+{
+    struct EndpointParams endpoint_params;
+    unsigned int dns_ttl_default;   ///< in seconds, DNS TTL when network request success
+    unsigned int dns_ttl_min;       ///< in seconds, DNS TTL when network request fail
+    int dns_threads;
+    int poller_threads;
+    int handler_threads;
+    int compute_threads;            ///< auto-set by system CPU number if value<=0
+};
+
+
+static constexpr struct WFGlobalSettings GLOBAL_SETTINGS_DEFAULT =
+{
+    .endpoint_params    =   ENDPOINT_PARAMS_DEFAULT,
+    .dns_ttl_default    =   12 * 3600,
+    .dns_ttl_min        =   180,
+    .dns_threads        =   4,
+    .poller_threads     =   4,
+    .handler_threads    =   20,
+    .compute_threads    =   -1,
+};
+~~~
+其中EndpointParams结构体和默认值在[EndpointParams.h](../src/manager/EndpointParams.h)文件里：
+~~~cpp
+
+struct EndpointParams
+{
+    size_t max_connections;
+    int connect_timeout;
+    int response_timeout;
+    int ssl_connect_timeout;
+};
+
+static constexpr struct EndpointParams ENDPOINT_PARAMS_DEFAULT =
+{
+    .max_connections        = 200,
+    .connect_timeout        = 10 * 1000,
+    .response_timeout       = 10 * 1000,
+    .ssl_connect_timeout    = 10 * 1000,
+};
+~~~
+举个例子，把默认的连接超时改为5秒，dns默认ttl改为1小时，用于消息反序列化的poller线程增加到10个：
+~~~cpp
+#include "workflow/WFGlobal.h"
+
+int main()
+{
+    struct WFGlobalSettings settings = GLOBAL_SETTINGS_DEFAULT;
+
+    settings.endpoint_params.connect_timeout = 5 * 1000;
+    settings.dns_ttl_default = 3600;
+    settings.poller_threads = 10;
+    WORKFLOW_library_init(&settings);
+
+    ...
+}
+
+~~~
+大多数参数的意义都比较清晰。注意dns ttl相关参数，单位是**秒**。endpoint相关超时参数单位是**毫秒**，并且可以用-1表示无限。  
+dns_threads表示并行访问dns的线程数。目前我们dns是通过系统函数getaddrinfo访问的。如果对dns并发性能有需求，可增加这个值。  
+compute_threads表示用于计算的线程数，默认-1代表与当前节点CPU核数相同。  
+与网络性能相关的两个参数为poller_threads和handler_threads：
+* poller线程主要负责epoll（kqueue）和消息反序列化。
+* handler线程是网络任务callback和process所在线程。
+
+所有框架需要的资源，都是在第一次被使用时才申请的。例如用户没有用到dns解析，那么dns线程不会被启动。  

docs/about-dns.md

@@ -40,15 +40,15 @@ static constexpr struct WFGlobalSettings GLOBAL_SETTING_DEFAULT =
     .endpoint_params    =    ENDPOINT_PARAMS_DEFAULT,
     .dns_ttl_default    =    12 * 3600,  /* in seconds */
     .dns_ttl_min        =    180,        /* reacquire when communication error */
-    .dns_threads        =    8,
-    .poller_threads     =    2,
+    .dns_threads        =    4,
+    .poller_threads     =    4,
     .handler_threads    =    20,
     .compute_threads    =    -1
 };
 //compute_threads<=0 means auto-set by system cpu number
 ~~~
 其中，与DNS相关的配置包括：
-  * dns_threads: DNS线程池线程数，默认8。
+  * dns_threads: DNS线程池线程数，默认4。
   * dns_ttl_default: DNS Cache中默认的TTL，单位秒，默认12小时，dns cache是当前进程的，即进程退出就会消失，配置也仅对当前进程有效。
   * dns_ttl_min: dns最短生效时间，单位秒，默认3分钟，用于通信失败重试是否尝试重新dns的决策。
 

docs/about-timeout.md

@@ -97,7 +97,7 @@ class WFNetworkTask : public CommRequest
 {
 ...
 public:
-    /* All in milleseconds. timeout == -1 for unlimited. */
+    /* All in milliseconds. timeout == -1 for unlimited. */
     void set_send_timeout(int timeout) { this->send_timeo = timeout; }
     void set_receive_timeout(int timeout) { this->receive_timeo = timeout; }
     void set_keep_alive(int timeout) { this->keep_alive_timeo = timeout; }

docs/benchmark.md

@@ -0,0 +1,211 @@
+# 性能测试
+
+Sogou C++ Workflow是一款性能优异的网络框架，本文介绍我们进行的性能测试，
+包括方案、代码、结果，以及与其他同类产品的对比。
+
+更多场景下的实验正在进行中，本文将持续更新。
+
+## HTTP Server
+
+HTTP Client/Server是Sogou C++ Workflow常见的应用场景，
+我们首先对Server端进行实验。
+
+### 环境
+
+我们部署了两台相同机器作为Server和Client，软硬件配置如下：
+
+| 软硬件 | 配置 |
+|:---:|:---|
+| CPU | 40 Cores, x86_64, Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz |
+| Memory | 192GB |
+| NIC | 25000Mbps |
+| OS | CentOS 7.8.2003 |
+| Kernel | Linux version 3.10.0-1127.el7.x86_64 |
+| GCC | 4.8.5 |
+
+两者间`ping`测得的RTT为0.1ms左右。
+
+### 对照组
+
+我们选择nginx和brpc作为对照组。
+选择前者是因为它在生产中部署十分广泛，性能不俗；
+对于后者，我们在本次实验中只关注HTTP Server方面的能力，
+其他的特性已有[单独的实验][Sogou RPC Benchmark]进行更为详尽的测试。
+
+事实上，我们也对此二者之外的其他某些框架同时进行了实验，
+但结果其性能表现相差较远，因此未在本文中体现。
+
+后续我们将选取更多合适的框架加入对比测试中。
+
+### Client工具
+
+本次实验我们使用的压测工具为[wrk][wrk]和[wrk2][wrk2]。
+前者适合测试特定并发下的QPS极限和延时，
+后者适合在特定QPS下测试延时分布。
+
+我们也尝试过使用其他测试工具，例如[ab][ab]等，但无法打出足够的压力。
+有鉴于此，我们也在着手开发基于Sogou C++ Workflow的benchmark工具。
+
+### 变量和指标
+
+一般而言，对网络框架的性能测试，切入的角度可谓纷繁多样。
+通过控制不同的变量、观测不同的指标，可以探究程序在不同场景下的适应能力。
+
+本次实验，我们选择其中最普遍常见的变量和指标：
+通过控制Client并发度和承载数据的大小，来测试QPS和延时的变化情况。
+另外，我们还测试了在掺杂慢请求的正常请求的延时分布。
+
+下面依次介绍两个测试场景。
+
+### 不同并发度和数据长度下的QPS和延时
+
+#### 代码和配置
+
+我们搭建了一个极其简约的HTTP服务器，
+忽略掉所有的业务逻辑，
+将测试点聚焦在纯粹的网络框架性能上。
+
+代码片段如下，
+完整代码移步[这里][benchmark-01 Code]。
+
+```cpp
+// ...
+
+auto * resp = task->get_resp();
+resp->add_header_pair("Date", timestamp);
+resp->add_header_pair("Content-Type", "text/plain; charset=UTF-8");
+resp->append_output_body_nocopy(content.data(), content.size());
+
+// ...
+```
+
+可以从上述代码中看到，
+对于到来的任何HTTP请求，
+我们都会返回一段固定的内容作为Body，
+并设置必要的Header，
+包括代码中指明的`content-type`、`date`，
+以及自动填充的`connection`和`content-length`。
+
+HTTP Body的固定内容是在Server启动时随机生成的ASCII字符串，
+其长度可以通过启动参数配置。
+同时可以配置的还有使用的poller线程数和监听的端口号。
+前者我们在本次测试中固定为16，
+因此Sogou C++ Workflow将使用16个poller线程和20个handler线程（默认配置）。
+
+对于nginx和brpc，
+我们也构建了相同的返回内容，
+并为nginx配置了40个进程、
+brpc配置了40个线程。
+
+
+#### 变量
+
+我们控制并发度在`[1, 2K]`之间翻倍增长，
+数据长度在`[16B, 64KB]`之间翻倍增长，
+两者正交。
+
+#### 指标
+
+鉴于并发度和数据长度组合之后数量较多，
+我们选择其中部分数据绘制为曲线。
+
+##### 固定数据长度下QPS与并发度关系
+
+![Concurrency and QPS][Con-QPS]
+
+上图可以看出，当数据长度保持不变，
+QPS随着并发度提高而增大，后趋于平稳。
+此过程中Sogou C++ Workflow一直有明显优势，
+高于brpc和nginx。
+特别是数据长度为64和512的两条曲线，
+并发度足够的时候，可以保持500K的QPS。
+
+注意上图中nginx-64与nginx-512的曲线重叠度很高，
+不易辨识。
+
+##### 固定并发度下QPS与数据长度关系
+
+![Body Length and QPS][Len-QPS]
+
+上图可以看出，当并发度保持不变，
+随着数据长度的增长，
+QPS保持平稳至4K时下降。
+此过程中，Sogou C++ Workflow也一直保持优势。
+
+##### 固定数据长度下延时与并发度关系
+
+![Concurrency and Latency][Con-Lat]
+
+上图可以看出，保持数据长度不变，
+延时随并发度提高而有所上升。
+此过程中，Sogou C++ Workflow略好于brpc，
+大好于nginx。
+
+##### 固定并发度下延时与数据长度关系
+
+![Body Length and Latency][Len-Lat]
+
+上图可以看出，并发度保持不变时，
+增大数据长度，造成延时上升。
+此过程中，Sogou C++ Workflow好于nginx，
+好于brpc。
+
+### 掺杂慢请求的延时分布
+
+#### 代码
+
+我们在上一个测试的基础上，简单添加了一个慢请求的逻辑，
+模拟业务场景中可能出现的特殊情况。
+
+代码片段如下，
+完整代码请移步[这里][benchmark-02 Code]。
+
+```cpp
+// ...
+
+if (std::strcmp(uri, "/long_req/") == 0)
+{
+    auto timer_task = WFTaskFactory::create_timer_task(microseconds, nullptr);
+    series_of(task)->push_back(timer_task);
+}
+// ...
+```
+
+我们在Server的process里进行判断，
+如果访问的是特定的路径，
+则添加一个`WFTimerTask`到Series的末尾，
+能够模拟一个异步耗时处理过程。
+类似地，对brpc使用`bthread_usleep()`函数进行异步睡眠。
+
+#### 配置
+
+在本次实验中，我们固定并发度为1024，数据长度为1024字节，
+分别以QPS为20K、100K和200K进行正常请求测试，
+测绘延时；
+与此同时，有另一路压力，进行慢请求，
+QPS是上述QPS的1%，
+数据不计入统计。
+慢请求的时长固定为5ms。
+
+#### 延时CDF图
+
+![Latency CDF][Lat CDF]
+
+从上图可以看出，当QPS为20K时，
+Sogou C++ Workflow略次于brpc；
+当QPS为100K时，两者几乎相当；
+当QPS为200K时，Sogou C++ Workflow略好于brpc。
+总之，可以认为两者在这方面旗鼓相当。
+
+
+[Sogou RPC Benchmark]: https://github.com/holmes1412/sogou-rpc-benchmark
+[wrk]: https://github.com/wg/wrk
+[wrk2]: https://github.com/giltene/wrk2
+[ab]: https://httpd.apache.org/docs/2.4/programs/ab.html
+[benchmark-01 Code]: ../benchmark/benchmark-01-http_server.cc
+[benchmark-02 Code]: ../benchmark/benchmark-02-http_server_long_req.cc
+[Con-QPS]: img/benchmark-01.png
+[Len-QPS]: img/benchmark-02.png
+[Con-Lat]: img/benchmark-03.png
+[Len-Lat]: img/benchmark-04.png
+[Lat CDF]: img/benchmark-05.png

src/client/WFMySQLConnection.h

@@ -30,6 +30,8 @@ public:
 	 * address resovled will be used. Don't use upstream name as a host. */
 	int init(const std::string& url);
 
+	void deinit() { }
+
 public:
 	WFMySQLTask *create_query_task(const std::string& query,
 								   mysql_callback_t callback);

src/factory/WFTask.h

@@ -274,7 +274,7 @@ public:
 	virtual WFConnection *get_connection() const;
 
 public:
-	/* All in milleseconds. timeout == -1 for unlimited. */
+	/* All in milliseconds. timeout == -1 for unlimited. */
 	void set_send_timeout(int timeout) { this->send_timeo = timeout; }
 	void set_receive_timeout(int timeout) { this->receive_timeo = timeout; }
 	void set_keep_alive(int timeout) { this->keep_alive_timeo = timeout; }
@@ -406,7 +406,7 @@ public:
 public:
 	ARGS *get_args() { return &this->args; }
 
-	long get_retval()
+	long get_retval() const
 	{
 		if (this->state == WFT_STATE_SUCCESS)
 			return this->get_res();

src/kernel/Communicator.cc

@@ -1804,7 +1804,7 @@ void Communicator::io_unbind(IOService *service)
 
 	if (mpoller_del(service->pipe_fd[0], this->mpoller) < 0)
 	{
-		/* Error occurred on event_fd or Communicator::deinit() called. */
+		/* Error occurred on pipe_fd or Communicator::deinit() called. */
 		this->shutdown_io_service(service);
 		errno = errno_bak;
 	}

src/kernel/IOService_linux.h

@@ -45,7 +45,7 @@ protected:
 	void prep_fdsync(int fd);
 
 protected:
-	long get_res() { return this->res; }
+	long get_res() const { return this->res; }
 
 private:
 	char iocb_buf[64];

src/kernel/IOService_thread.h

@@ -46,7 +46,7 @@ protected:
 	void prep_fdsync(int fd);
 
 protected:
-	long get_res() { return this->res; }
+	long get_res() const { return this->res; }
 
 private:
 	int fd;

src/kernel/poller.c

@@ -66,11 +66,13 @@ struct __poller_node
 	int state;
 	int error;
 	struct poller_data data;
+#pragma pack(1)
 	union
 	{
 		struct list_head list;
 		struct rb_node rb;
 	};
+#pragma pack()
 	char in_rbtree;
 	char removed;
 	int event;
@@ -959,7 +961,7 @@ static void __poller_handle_notify(struct __poller_node *node,
 		node->state = PR_ST_ERROR;
 	}
 
-	__poller_add_result(res, poller);
+	__poller_add_result(node, poller);
 }
 
 static int __poller_handle_pipe(poller_t *poller)

src/manager/WFGlobal.cc

@@ -40,8 +40,6 @@
 #include "WFTask.h"
 #include "WFTaskError.h"
 
-const char *library_version = "Workflow Library Version: 1.12.6";
-
 class __WFGlobal
 {
 public:

src/manager/WFGlobal.h

@@ -61,8 +61,8 @@ static constexpr struct WFGlobalSettings GLOBAL_SETTINGS_DEFAULT =
 	.endpoint_params	=	ENDPOINT_PARAMS_DEFAULT,
 	.dns_ttl_default	=	12 * 3600,
 	.dns_ttl_min		=	180,
-	.dns_threads		=	8,
-	.poller_threads		=	2,
+	.dns_threads		=	4,
+	.poller_threads		=	4,
 	.handler_threads	=	20,
 	.compute_threads	=	-1,
 };

src/protocol/RedisMessage.cc

@@ -475,7 +475,7 @@ std::string RedisValue::debug_string() const
 		for (size_t i = 0; i < l; i++)
 		{
 			if (i)
-				ret += " ,";
+				ret += ", ";
 
 			ret += (*this)[i].debug_string();
 		}

src/server/WFServer.h

@@ -63,10 +63,10 @@ public:
 public:
 	/* To start a TCP server */
 
-	/* Start on port. On linux, IPv4 is default. */
+	/* Start on port with IPv4. */
 	int start(unsigned short port)
 	{
-		return start(AF_UNSPEC, NULL, port, NULL, NULL);
+		return start(AF_INET, NULL, port, NULL, NULL);
 	}
 
 	/* Start with family. AF_INET or AF_INET6. */
@@ -78,7 +78,7 @@ public:
 	/* Start with hostname and port. */
 	int start(const char *host, unsigned short port)
 	{
-		return start(AF_UNSPEC, host, port, NULL, NULL);
+		return start(AF_INET, host, port, NULL, NULL);
 	}
 
 	/* Start with family, hostname and port. */
@@ -97,7 +97,7 @@ public:
 
 	int start(unsigned short port, const char *cert_file, const char *key_file)
 	{
-		return start(AF_UNSPEC, NULL, port, cert_file, key_file);
+		return start(AF_INET, NULL, port, cert_file, key_file);
 	}
 
 	int start(int family, unsigned short port,
@@ -109,7 +109,7 @@ public:
 	int start(const char *host, unsigned short port,
 			  const char *cert_file, const char *key_file)
 	{
-		return start(AF_UNSPEC, host, port, cert_file, key_file);
+		return start(AF_INET, host, port, cert_file, key_file);
 	}
 
 	int start(int family, const char *host, unsigned short port,
@@ -134,7 +134,7 @@ public:
 		this->wait_finish();
 	}
 
-	/* Nonblocking terminating the server. For stopping multiply servers.
+	/* Nonblocking terminating the server. For stopping multiple servers.
 	 * Typically, call shutdown() and then wait_finish().
 	 * But indeed wait_finish() can be called before shutdown(), even before
 	 * start() in another thread. */

tutorial/tutorial-12-mysql_cli.cc

@@ -81,7 +81,9 @@ void mysql_callback(WFMySQLTask *task)
 
 	if (task->get_state() != WFT_STATE_SUCCESS)
 	{
-		fprintf(stderr, "task error = %d\n", task->get_error());
+		fprintf(stderr, "error msg: %s\n",
+				WFGlobal::get_error_string(task->get_state(),
+										   task->get_error()));
 		return;
 	}