原文地址 Development guidey
阅读这篇英文doc和阅读英文rfc 一样，基础，作用不大，小例子，读起来缓慢.中文的分析blog都是基于这篇guide的,列举的各种部分和元素对自己设计一个系统有帮助。
整体设计：数据结构设计和内存结构
内存池，线程池，连接池
模块:插件式 &module固定steps
配置：server,http,location 3层
进程：master +多个worker
网络(外部通信层):主进程listen&子进程accept metux+epoll实现高并发
Event（内部通信）:系统内消息机制(放入queue)
handler（内部通信）:callback
http（业务逻辑）：http,反向代理，subrequests 3种处理. 每种处理分成 request解析,respond解析.状态机8步接收，11个阶段处理
log机制
Each HTTP client connection runs through the following stages:
- ngx_event_accept() accepts a client TCP connection. For HTTP connections it's ngx_http_init_connection(c).
- ngx_http_init_connection() performs early initialization of the HTTP connection. At this stage an ngx_http_connection_t object is created for the connection and its reference is stored in the connection's data field. Later it will be replaced by an HTTP request object. A PROXY protocol parser and the SSL handshake are started at this stage as well.
- ngx_http_wait_request_handler() read event handler is called when data is available on the client socket. At this stage an HTTP request object ngx_http_request_t is created and set to the connection's data field.
- ngx_http_process_request_line() read event handler reads client request line. The handler is set by ngx_http_wait_request_handler(). The data is read into connection's buffer. The size of the buffer is initially set by the directive client_header_buffer_size.
- ngx_http_process_request_headers() read event handler, is set after ngx_http_process_request_line() to read the client request header.
- ngx_http_core_run_phases() is called when the request header is completely read and parsed. This function runs request phases from NGX_HTTP_POST_READ_PHASE to NGX_HTTP_CONTENT_PHASE. The last phase is intended to generate a response and pass it along the filter chain. The response is not necessarily sent to the client at this phase.
- ngx_http_finalize_request() is usually called when the request has generated all the output or produced an error.
- ngx_http_finalize_connection() is called when the complete response has been sent to the client and the request can be destroyed.
For each client HTTP request the ngx_http_request_t object is created.
Some of the fields of this object are:
-
connection — Pointer to a ngx_connection_t client connection object. Several requests can reference the same connection object at the same time - one main request and its subrequests.
Note that for HTTP connections ngx_connection_t's data field points back to the request. Such requests are called active, as opposed to the other requests tied to the connection.
An active request is used to handle client connection events and is allowed to output its response to the client. -
ctx — Array of HTTP module contexts. Each module of type NGX_HTTP_MODULE can store any value (normally, a pointer to a structure) in the request.
- ngx_http_get_module_ctx(r, module) — Returns the module's context
- ngx_http_set_ctx(r, c, module) — Sets c as the module's context
- main_conf, srv_conf, loc_conf — Arrays of current request configurations. Configurations are stored at the module's ctx_index positions.
- read_event_handler, write_event_handler - Read and write event handlers for the request. Normally, both the read and write event handlers for an HTTP connection are set to ngx_http_request_handler(). This function calls the read_event_handler and write_event_handler handlers for the currently active request.
- cache — Request cache object for caching the upstream response.
- upstream — Request upstream object for proxying.
- pool — Request pool. The request object itself is allocated in this pool, which is destroyed when the request is deleted.
- header_in — Buffer into which the client HTTP request header is read.
- headers_in, headers_out — Input and output HTTP headers objects. Both objects contain the headers field of type ngx_list_t for keeping the raw list of headers. In addition to that, specific headers are available for getting and setting as separate fields, for example content_length_n, status etc.
- request_body — Client request body object.
- start_sec, start_msec — Time point when the request was created, used for tracking request duration.
- method, method_name — Numeric and text representation of the client HTTP request method. Numeric values for methods are defined in src/http/ngx_http_request.h with the macros NGX_HTTP_GET, NGX_HTTP_HEAD, NGX_HTTP_POST, etc.
- http_protocol — Client HTTP protocol version in its original text form (“HTTP/1.0”, “HTTP/1.1” etc).
- http_version — Client HTTP protocol version in numeric form (NGX_HTTP_VERSION_10, NGX_HTTP_VERSION_11, etc.).
- http_major, http_minor — Client HTTP protocol version in numeric form split into major and minor parts.
- request_line, unparsed_uri — Request line and URI in the original client request.
- uri, args, exten — URI, arguments and file extension for the current request. The URI value here might differ from the original URI sent by the client due to normalization. Throughout request processing, these values can change as internal redirects are performed.
- main — Pointer to a main request object. This object is created to process a client HTTP request, as opposed to subrequests, which are created to perform a specific subtask within the main request.
- parent — Pointer to the parent request of a subrequest.
- postponed — List of output buffers and subrequests, in the order in which they are sent and created. The list is used by the postpone filter to provide consistent request output when parts of it are created by subrequests.
- post_subrequest — Pointer to a handler with the context to be called when a subrequest gets finalized. Unused for main requests.
-
posted_requests — List of requests to be started or resumed, which is done by calling the request's write_event_handler. Normally, this handler holds the request main function, which at first runs request phases and then produces the output.
A request is usually posted by the ngx_http_post_request(r, NULL) call.
It is always posted to the main request posted_requests list.
The function ngx_http_run_posted_requests(c) runs all requests that are posted in the main request of the passed connection's active request.
All event handlers call ngx_http_run_posted_requests,which lead to new posted requests
Normally, it is called after invoking a request's read or write handler. - phase_handler — Index of current request phase.
- subrequests — Current subrequest nesting level. Each subrequest inherits its parent's nesting level, decreased by one.
Each HTTP module can have three types of configuration:
- Main configuration — Applies to the entire http block. Functions as global settings for a module.
- Server configuration — Applies to a single server block. Functions as server-specific settings for a module.
- Location configuration — Applies to a single location, if or limit_except block. Functions as location-specific settings for a module.
Configuration structures are created at the nginx configuration stage by calling functions, which allocate the structures, initialize them and merge them.
The following example shows how to create a simple location configuration for a module.
The configuration has one setting, foo, of type unsigned integer.
typedef struct {
ngx_uint_t foo;
} ngx_http_foo_loc_conf_t;
static ngx_http_module_t ngx_http_foo_module_ctx = {
NULL,
......
ngx_http_foo_create_loc_conf,
ngx_http_foo_merge_loc_conf
};
static void *
ngx_http_foo_create_loc_conf(ngx_conf_t *cf)
{
ngx_http_foo_loc_conf_t *conf;
conf = ngx_pcalloc(cf->pool, sizeof(ngx_http_foo_loc_conf_t));
conf->foo = NGX_CONF_UNSET_UINT;
return conf;
}
static char *
ngx_http_foo_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child)
{
ngx_http_foo_loc_conf_t *prev = parent;
ngx_http_foo_loc_conf_t *conf = child;
ngx_conf_merge_uint_value(conf->foo, prev->foo, 1);
}
As seen in the example, the ngx_http_foo_create_loc_conf() function creates a new configuration structure, and ngx_http_foo_merge_loc_conf() merges a configuration with configuration from a higher level.
The following macros are available. for accessing configuration for HTTP modules at configuration time. They all take ngx_conf_t reference as the first argument.
- ngx_http_conf_get_module_main_conf(cf, module)
- ngx_http_conf_get_module_srv_conf(cf, module)
- ngx_http_conf_get_module_loc_conf(cf, module)
The following example gets a pointer to a location configuration of standard nginx core module ngx_http_core_module and replaces the location content handler kept in the handler field of the structure.
static ngx_int_t ngx_http_foo_handler(ngx_http_request_t *r);
static ngx_command_t ngx_http_foo_commands[] = {
{ ngx_string("foo"),
NGX_HTTP_LOC_CONF|NGX_CONF_NOARGS,
ngx_http_foo,
0,
0,
NULL },
ngx_null_command
};
static char *
ngx_http_foo(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
{
ngx_http_core_loc_conf_t *clcf;
clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
clcf->handler = ngx_http_bar_handler;
return NGX_CONF_OK;
}
Event Event object ngx_event_t in nginx provides a mechanism for notification that a specific event has occurred.
data — Arbitrary event context used in event handlers, usually as pointer to a connection related to the event.
handler — Callback function to be invoked when the event happens.
write — Flag indicating a write event. Absence of the flag indicates a read event.
active —Flag indicating that the event is registered for receiving I/O notifications, normally from notification mechanisms like epoll, kqueue, poll
ready — Flag indicating that the event has received an I/O notification.
delayed — Flag indicating that I/O is delayed due to rate limiting.
timer — Red-black tree node for inserting the event into the timer tree.
timer_set — Flag indicating that the event timer is set and not yet expired.
timedout — Flag indicating that the event timer has expired.
eof — Flag indicating that EOF occurred while reading data.
pending_eof — Flag indicating that EOF is pending on the socket
error — Flag indicating that an error occurred during reading (for a read event) or writing (for a write event).
cancelable — Timer event flag indicating that the event should be ignored while shutting down the worker.
posted — Flag indicating that the event is posted to a queue.
queue — Queue node for posting the event to a queue.
NGX_PROCESS_MASTER — The master process, which reads the NGINX configuration, creates cycles, and starts and controls child processes.
It does not perform any I/O and responds only to signals. Its cycle function is ngx_master_process_cycle().
NGX_PROCESS_WORKER — The worker process, which handles client connections.
It is started by the master process and responds to its signals and channel commands as well.
Its cycle function is ngx_worker_process_cycle().
typedef struct {
int foo;
} my_thread_ctx_t;
static void
my_thread_func(void *data, ngx_log_t *log)
{
my_thread_ctx_t *ctx = data;
}
static void
my_thread_completion(ngx_event_t *ev)
{
my_thread_ctx_t *ctx = ev->data;
}
ngx_int_t
my_task_offload(my_conf_t *conf)
{
my_thread_ctx_t *ctx;
ngx_thread_task_t *task;
task = ngx_thread_task_alloc(conf->pool, sizeof(my_thread_ctx_t));
ctx = task->ctx;
ctx->foo = 42;
task->handler = my_thread_func;
task->event.handler = my_thread_completion;
task->event.data = ctx;
if (ngx_thread_task_post(conf->thread_pool, task) != NGX_OK) {
return NGX_ERROR;
}
return NGX_OK;
}
Modules
Adding new modules
Each standalone nginx module resides in a separate directory that contains at least two files: config and a file with the module source code.
The config file contains all information needed for nginx to integrate the module, for example:
ngx_module_type=CORE
ngx_module_name=ngx_foo_module
ngx_module_srcs="$ngx_addon_dir/ngx_foo_module.c"
. auto/module
ngx_addon_name=$ngx_module_name
The config file is a POSIX shell script that can set and access the following variables:
ngx_module_type — Type of module to build. Possible values are CORE, HTTP, HTTP_FILTER, HTTP_INIT_FILTER, HTTP_AUX_FILTER, MAIL, STREAM, or MISC.
....
Core Modules
Modules are the building blocks of nginx, and most of its functionality is implemented as modules.
The module source file must contain a global variable of type ngx_module_t, which is defined as follows:
Each module keeps its private data in the ctx field, recognizes the configuration directives
The module type defines exactly what is stored in the ctx field. Its value is one of the following types:
NGX_CORE_MODULE
NGX_EVENT_MODULE
NGX_HTTP_MODULE
NGX_MAIL_MODULE
NGX_STREAM_MODULE
typedef struct {
ngx_str_t name;
void *(*create_conf)(ngx_cycle_t *cycle);
char *(*init_conf)(ngx_cycle_t *cycle, void *conf);
} ngx_core_module_t;
where the name is a module name string, create_conf and init_conf are pointers to functions that create and initialize module configuration respectively.
For core modules, nginx calls create_conf before parsing a new configuration and init_conf after all configuration is parsed successfully.
The typical create_conf function allocates memory for the configuration and sets default values.
For example, a simplistic module called ngx_foo_module might look like this:
#includeConfiguration Directives#include typedef struct { ngx_flag_t enable; } ngx_foo_conf_t; static void *ngx_foo_create_conf(ngx_cycle_t *cycle); static char *ngx_foo_init_conf(ngx_cycle_t *cycle, void *conf); static ngx_command_t ngx_foo_commands[] = { { ngx_string("foo_enabled"), NGX_MAIN_CONF|NGX_DIRECT_CONF|NGX_CONF_FLAG, ngx_conf_set_flag_slot, 0, offsetof(ngx_foo_conf_t, enable), &ngx_foo_enable_post }, ngx_null_command }; static ngx_core_module_t ngx_foo_module_ctx = { ngx_string("foo"), ngx_foo_create_conf, ngx_foo_init_conf }; ngx_module_t ngx_foo_module = { NGX_MODULE_V1, &ngx_foo_module_ctx, ngx_foo_commands, NGX_CORE_MODULE, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NGX_MODULE_V1_PADDING }; static void * ngx_foo_create_conf(ngx_cycle_t *cycle) { ngx_foo_conf_t *fcf; fcf = ngx_pcalloc(cycle->pool, sizeof(ngx_foo_conf_t)); fcf->enable = NGX_CONF_UNSET; return fcf; } static char * ngx_foo_init_conf(ngx_cycle_t *cycle, void *conf) { ngx_foo_conf_t *fcf = conf; ngx_conf_init_value(fcf->enable, 0); return NGX_CONF_OK; }
The ngx_command_t type defines a single configuration directive. Each module that supports configuration provides an array of such structures that describe how to process arguments and what handlers to call:
typedef struct ngx_command_s ngx_command_t;
struct ngx_command_s {
ngx_str_t name;
ngx_uint_t type;
char *(*set)(ngx_conf_t *cf, ngx_command_t *cmd, void *conf);
ngx_uint_t conf;
ngx_uint_t offset;
void *post;
};
The name is the name of a directive as it appears in the configuration file, for example "worker_processes" or "listen". The type is a bit-field of flags that specify the number of arguments the directive takes, its type, and the context in which it appears. The flags are:
NGX_HTTP_MAIN_CONF_OFFSET — Configuration for the http block.
NGX_HTTP_SRV_CONF_OFFSET — Configuration for a server block within the http block.
NGX_HTTP_LOC_CONF_OFFSET — Configuration for a location block within the http.
.....
The offset defines the offset of a field in a module configuration structure that holds values for this particular directive. The typical use is to employ the offsetof() macro.
The post field has two purposes: it may be used to define a handler to be called after the main handler has completed, or to pass additional data to the main handler.
HTTPEach HTTP client connection runs through the following stages:
- ngx_event_accept() accepts a client TCP connection.
- ngx_http_init_connection() performs early initialization of the HTTP connection. At this stage an ngx_http_connection_t object is created for the connection and its reference is stored in the connection's data field. Later it will be replaced by an HTTP request object. A PROXY protocol parser and the SSL handshake are started at this stage as well.
- ngx_http_wait_request_handler() read event handler is called when data is available on the client socket. At this stage an HTTP request object ngx_http_request_t is created and set to the connection's data field.
- ngx_http_process_request_line() read event handler reads client request line. The handler is set by ngx_http_wait_request_handler().
- ngx_http_process_request_headers() read event handler, is set after ngx_http_process_request_line() to read the client request header.
- ngx_http_core_run_phases() is called when the request header is completely read and parsed. This function runs request phases from NGX_HTTP_POST_READ_PHASE to NGX_HTTP_CONTENT_PHASE. The last phase is intended to generate a response and pass it along the filter chain.
- ngx_http_finalize_request() is usually called when the request has generated all the output or produced an error.
- ngx_http_finalize_connection() is called when the complete response has been sent to the client and the request can be destroyed.
For each client HTTP request the ngx_http_request_t object is created.
Some of the fields of this object are:
connection — Pointer to a ngx_connection_t client connection object.
Several requests can reference the same connection object at the same time - one main request and its subrequests.
Note that for HTTP connections ngx_connection_t's data field points back to the request.
ctx — Array of HTTP module contexts.
The following macros provide a convenient way to get and set request contexts:
ngx_http_get_module_ctx(r, module) — Returns the module's context
ngx_http_set_ctx(r, c, module) — Sets c as the module's context
main_conf, srv_conf, loc_conf — Arrays of current request configurations.
read_event_handler, write_event_handler - Read and write event handlers for the request. Normally, both the read and write event handlers for an HTTP connection are set to ngx_http_request_handler().
cache — Request cache object for caching the upstream response.
upstream — Request upstream object for proxying.
pool — Request pool.
header_in — Buffer into which the client HTTP request header is read.
headers_in, headers_out — Input and output HTTP headers objects.
request_body — Client request body object.
start_sec, start_msec — Time point when the request was created, used for tracking request duration.
method, method_name — Numeric and text representation of the client HTTP request method. eg:NGX_HTTP_GET, NGX_HTTP_HEAD, NGX_HTTP_POST, etc.
http_protocol— Client HTTP protocol version in its original text form (“HTTP/1.0”, “HTTP/1.1” etc).
http_version — Client HTTP protocol version in numeric form (NGX_HTTP_VERSION_10, NGX_HTTP_VERSION_11, etc.).
http_major, http_minor — Client HTTP protocol version in numeric form split into major and minor parts.
request_line, unparsed_uri — Request line and URI in the original client request.
uri, args, exten — URI, arguments and file extension for the current request.
main — Pointer to a main request object. This object is created to process a client HTTP request, as opposed to subrequests, which are created to perform a specific subtask within the main request.
parent — Pointer to the parent request of a subrequest.
postponed — List of output buffers and subrequests, in the order in which they are sent and created. The list is used by the postpone filter to provide consistent request output when parts of it are created by subrequests.
post_subrequest — Pointer to a handler with the context to be called when a subrequest gets finalized.
posted_requests — List of requests to be started or resumed, which is done by calling the request's write_event_handler. Normally, this handler holds the request main function, which at first runs request phases and then produces the output.
A request is usually posted by the ngx_http_post_request(r, NULL) call. It is always posted to the main request posted_requests list.
The function ngx_http_run_posted_requests(c) runs all requests that are posted in the main request of the passed connection's active request.
All event handlers call ngx_http_run_posted_requests, which can lead to new posted requests.
Normally, it is called after invoking a request's read or write handler.
phase_handler — Index of current request phase.
subrequests — Current subrequest nesting level.
blocked — Counter of blocks held on the request. While this value is non-zero, the request cannot be terminated. Currently, this value is increased by pending AIO operations (POSIX AIO and thread operations) and active cache lock.
buffered — Bitmask showing which modules have buffered the output produced by the request. A number of filters can buffer output;
for example, sub_filter can buffer data because of a partial string match, copy filter can buffer data because of the lack of free output buffers etc. As long as this value is non-zero, the request is not finalized pending the flush.
header_only — Flag indicating that the output does not require a body. For example, this flag is used by HTTP HEAD requests.
keepalive — Flag indicating whether client connection keepalive is supported. The value is inferred from the HTTP version and the value of the “Connection” header.
Configuration
Each HTTP module can have three types of configuration:
Main configuration — Applies to the entire http block. Functions as global settings for a module.
Server configuration — Applies to a single server block. Functions as server-specific settings for a module.
Location configuration — Applies to a single location, if or limit_except block. Functions as location-specific settings for a module.
Configuration structures are created at the nginx configuration stage by calling functions, which allocate the structures, initialize them and merge them.
The following example shows how to create a simple location configuration for a module. The configuration has one setting, foo, of type unsigned integer.
typedef struct {
ngx_uint_t foo;
} ngx_http_foo_loc_conf_t;
static ngx_http_module_t ngx_http_foo_module_ctx = {
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
ngx_http_foo_create_loc_conf,
ngx_http_foo_merge_loc_conf
};
static void *
ngx_http_foo_create_loc_conf(ngx_conf_t *cf)
{
ngx_http_foo_loc_conf_t *conf;
conf = ngx_pcalloc(cf->pool, sizeof(ngx_http_foo_loc_conf_t));
conf->foo = NGX_CONF_UNSET_UINT;
return conf;
}
static char *
ngx_http_foo_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child)
{
ngx_http_foo_loc_conf_t *prev = parent;
ngx_http_foo_loc_conf_t *conf = child;
ngx_conf_merge_uint_value(conf->foo, prev->foo, 1);
}
As seen in the example, the ngx_http_foo_create_loc_conf() function creates a new configuration structure, and ngx_http_foo_merge_loc_conf() merges a configuration with configuration from a higher level.
Specifically, a server configuration is also created at the main level and location configurations are created at the main, server, and location levels.
static ngx_int_t ngx_http_foo_handler(ngx_http_request_t *r);
static ngx_command_t ngx_http_foo_commands[] = {
{ ngx_string("foo"),
NGX_HTTP_LOC_CONF|NGX_CONF_NOARGS,
ngx_http_foo,
0,
0,
NULL },
ngx_null_command
};
static char *
ngx_http_foo(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
{
ngx_http_core_loc_conf_t *clcf;
clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
clcf->handler = ngx_http_foo_handler;
return NGX_CONF_OK;
}
static ngx_int_t
ngx_http_foo_handler(ngx_http_request_t *r)
{
ngx_http_foo_loc_conf_t *flcf;
flcf = ngx_http_get_module_loc_conf(r, ngx_http_foo_module);
...
}
Phases 11个阶段
Each HTTP request passes through a sequence of phases. In each phase a distinct type of processing is performed on the request.
Following is the list of nginx HTTP phases.
- 1 NGX_HTTP_POST_READ_PHASE — First phase. The ngx_http_realip_module registers its handler at this phase to enable substitution of client addresses before any other module is invoked.
- 2 NGX_HTTP_SERVER_REWRITE_PHASE — Phase where rewrite directives defined in a server block
- 3 NGX_HTTP_FIND_CONFIG_PHASE
- 4 NGX_HTTP_REWRITE_PHASE
- 5 NGX_HTTP_POST_REWRITE_PHASE
- 6 NGX_HTTP_PREACCESS_PHASE
- 7 NGX_HTTP_ACCESS_PHASE — Phase where it is verified that the client is authorized to make the request. Standard nginx modules such as ngx_http_access_module and ngx_http_auth_basic_module register their handlers at this phase. By default the client must pass the authorization check of all handlers registered at this phase for the request to continue to the next phase. The satisfy directive, can be used to permit processing to continue if any of the phase handlers authorizes the client.
- 8 NGX_HTTP_POST_ACCESS_PHASE
- NGX_HTTP_CONTENT_PHASE — Phase where the response is normally generated.
- 9 NGX_HTTP_PRECONTENT_PHASE
- 10 NGX_HTTP_CONTENT_PHASE
- 11 NGX_HTTP_LOG_PHASE
Following is the example of a preaccess phase handler.
static ngx_http_module_t ngx_http_foo_module_ctx = {
NULL,
ngx_http_foo_init,
NULL,
NULL,
NULL,
NULL,
NULL,
NULL
};
static ngx_int_t
ngx_http_foo_handler(ngx_http_request_t *r)
{
ngx_str_t *ua;
ua = r->headers_in->user_agent;
if (ua->value.len == 3 && ngx_strncmp(ua->value.data, "foo", 3) == 0) {
return NGX_HTTP_FORBIDDEN;
}
return NGX_DECLINED;
}
static ngx_int_t
ngx_http_foo_init(ngx_conf_t *cf)
{
ngx_http_handler_pt *h;
ngx_http_core_main_conf_t *cmcf;
cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module);
h = ngx_array_push(&cmcf->phases[NGX_HTTP_PREACCESS_PHASE].handlers);
*h = ngx_http_foo_handler;
return NGX_OK;
}
Phase handlers are expected to return specific codes:
At the content phase, any return code other that NGX_DECLINED is considered a finalization code.Any return code from the location content handlers is considered a finalization code.
At the access phase, in satisfy any mode, any return code other than NGX_OK, NGX_DECLINED, NGX_AGAIN, NGX_DONE is considered a denial.
Request redirection
An HTTP request is always connected to a location via the loc_conf field of
the ngx_http_request_t structure.
The next change of the location takes place at the NGX_HTTP_FIND_CONFIG_PHASE
request phase.
The ngx_http_rewrite_module can change the request URI at the
NGX_HTTP_REWRITE_PHASE request phase as a result of the rewrite directive and
send the request back to the NGX_HTTP_FIND_CONFIG_PHASE phase for selection
of a new location based on the new URI.
It is also possible to redirect a request to a new location at any point by
calling one of ngx_http_internal_redirect(r, uri, args) or
ngx_http_named_location(r, name).
The ngx_http_internal_redirect(r, uri, args) function changes the request URI
and returns the request to the NGX_HTTP_SERVER_REWRITE_PHASE phase. The
request proceeds with a server default location. Later at
NGX_HTTP_FIND_CONFIG_PHASE a new location is chosen based on the new request
URI.
The following example performs an internal redirect with the new request arguments.
ngx_int_t
ngx_http_foo_redirect(ngx_http_request_t *r)
{
ngx_str_t uri, args;
ngx_str_set(&uri, "/foo");
ngx_str_set(&args, "bar=1");
return ngx_http_internal_redirect(r, &uri, &args);
}
The following example performs a redirect to a named location @foo.
ngx_int_t
ngx_http_foo_named_redirect(ngx_http_request_t *r)
{
ngx_str_t name;
ngx_str_set(&name, "foo");
return ngx_http_named_location(r, &name);
}
Request finalization
An HTTP request is finalized by calling the function ngx_http_finalize_request(r, rc). It is usually finalized by the content handler after all output buffers are sent to the filter chain. At this point all of the output might not be sent to the client, with some of it remaining buffered somewhere along the filter chain. If it is, the ngx_http_finalize_request(r, rc) function automatically installs a special handler ngx_http_writer(r) to finish sending the output. A request is also finalized in case of an error or if a standard HTTP response code needs to be returned to the client.
SubrequestsSubrequests are primarily used to insert output of one request into another,
possibly mixed with other data.
A subrequest looks like a normal request, but shares some data with its parent
In particular, all fields related to client input are shared because a
subrequest does not receive any other input from the client.
The request field parent for a subrequest contains a link to its parent
request and is NULL for the main request.
The field main contains a link to the main request in a group of requests.
A subrequest starts in the NGX_HTTP_SERVER_REWRITE_PHASE phase. It passes
through the same subsequent phases as a normal request and is assigned a
location based on its own URI.
The output header in a subrequest is always ignored. The
ngx_http_postpone_filter places the subrequest's output body in the right
position relative to other data produced by the parent request.
Subrequests are related to the concept of active requests.
Initially, the main request is active.
The first subrequest of an active request becomes active right after creation.
The ngx_http_postpone_filter activates the next request in the active request'
s subrequest list, once all data prior to that request are sent.
When a request is finalized, its parent is activated.
Create a subrequest by calling the function ngx_http_subrequest(r, uri, args,
psr, ps, flags), where r is the parent request, uri and args are the URI and
arguments of the subrequest, psr is the output parameter, which receives the
newly created subrequest reference, ps is a callback object for notifying the
parent request that the subrequest is being finalized, and flags is bitmask
of flags.
This example clones the current request and sets a finalization callback for
the subrequest.
ngx_int_t
ngx_http_foo_clone(ngx_http_request_t *r)
{
ngx_http_request_t *sr;
ngx_http_post_subrequest_t *ps;
ps = ngx_palloc(r->pool, sizeof(ngx_http_post_subrequest_t));
ps->handler = ngx_http_foo_subrequest_done;
ps->data = "foo";
return ngx_http_subrequest(r, &r->uri, &r->args, &sr, ps,
NGX_HTTP_SUBREQUEST_CLONE);
}
ngx_int_t
ngx_http_foo_subrequest_done(ngx_http_request_t *r, void *data, ngx_int_t rc)
{
char *msg = (char *) data;
return rc;
}
The following example inserts output from a subrequest after all request data
buffers, but before the final buffer with the last_buf flag.
ngx_int_t
ngx_http_foo_body_filter(ngx_http_request_t *r, ngx_chain_t *in)
{
ngx_int_t rc;
ngx_buf_t *b;
ngx_uint_t last;
ngx_chain_t *cl, out;
ngx_http_request_t *sr;
ngx_http_foo_filter_ctx_t *ctx;
ctx = ngx_http_get_module_ctx(r, ngx_http_foo_filter_module);
if (ctx == NULL) {
return ngx_http_next_body_filter(r, in);
}
last = 0;
for (cl = in; cl; cl = cl->next) {
if (cl->buf->last_buf) {
cl->buf->last_buf = 0;
cl->buf->last_in_chain = 1;
cl->buf->sync = 1;
last = 1;
}
}
rc = ngx_http_next_body_filter(r, in);
if (ngx_http_subrequest(r, ctx->uri, NULL, &sr, NULL, 0) != NGX_OK) {
return NGX_ERROR;
}
ngx_http_set_ctx(r, NULL, ngx_http_foo_filter_module);
b = ngx_calloc_buf(r->pool);
b->last_buf = 1;
out.buf = b;
out.next = NULL;
return ngx_http_output_filter(r, &out);
}
A subrequest can also be created for other purposes than data output.
For example, the ngx_http_auth_request_module module creates a subrequest at
the NGX_HTTP_ACCESS_PHASE phase.
To disable output at this point, the header_only flag is set on the subrequest
This prevents the subrequest body from being sent to the client.
Note that the subrequest's header is never sent to the client. The result of
the subrequest can be analyzed in the callback handler.
Request body
For dealing with the body of a client request, nginx provides the ngx_http_read_client_request_body(r, post_handler)
and ngx_http_discard_request_body(r) functions.
First function reads the request body and makes it available via the request_body request field.
Second function instructs nginx to discard (read and ignore) the request body.
One of these functions must be called for every request.
Normally, the content handler makes the call.
Reading or discarding the client request body from a subrequest is not allowed.
It must always be done in the main request.
When a subrequest is created, it inherits the parent's request_body object which can be used by the subrequest if the main request has previously read the request body.
The function ngx_http_read_client_request_body(r, post_handler) starts the process of reading the request body. When the body is completely read, the post_handler callback is called to continue processing the request. If the request body is missing or has already been read, the callback is called immediately.
The function ngx_http_read_client_request_body(r, post_handler) allocates the request_body request field of type ngx_http_request_body_t.
The following example reads a client request body and returns its size.
ngx_int_t
ngx_http_foo_content_handler(ngx_http_request_t *r)
{
ngx_int_t rc;
rc = ngx_http_read_client_request_body(r, ngx_http_foo_init);
return NGX_DONE;
}
void
ngx_http_foo_init(ngx_http_request_t *r)
{
off_t len;
ngx_buf_t *b;
ngx_int_t rc;
ngx_chain_t *in, out;
len = 0;
for (in = r->request_body->bufs; in; in = in->next) {
len += ngx_buf_size(in->buf);
}
b = ngx_create_temp_buf(r->pool, NGX_OFF_T_LEN);
b->last = ngx_sprintf(b->pos, "%O", len);
b->last_buf = (r == r->main) ? 1: 0;
b->last_in_chain = 1;
r->headers_out.status = NGX_HTTP_OK;
r->headers_out.content_length_n = b->last - b->pos;
rc = ngx_http_send_header(r);
out.buf = b;
out.next = NULL;
rc = ngx_http_output_filter(r, &out);
ngx_http_finalize_request(r, rc);
}
The return value NGX_AGAIN and the request flag reading_body indicate that more data is available.
If bufs is NULL after calling this function, there is nothing to read at the moment. The request callback read_event_handler will be called when the next part of request body is available.
Request body filters
After a request body part is read, it's passed to the request body filter chain by calling the first body filter handler stored in the ngx_http_top_request_body_filter variable.
It's assumed that every body handler calls the next handler in the chain until the final handler ngx_http_request_body_save_filter(r, cl) is called. This handler collects the buffers in r->request_body->bufs and writes them to a file if necessary. The last request body buffer has nonzero last_buf flag.
Following is an example of a simple request body filter that delays request body by one second.
#define NGX_HTTP_DELAY_BODY 1000
typedef struct {
ngx_event_t event;
ngx_chain_t *out;
} ngx_http_delay_body_ctx_t;
static ngx_int_t ngx_http_delay_body_filter(ngx_http_request_t *r,
ngx_chain_t *in);
static void ngx_http_delay_body_cleanup(void *data);
static void ngx_http_delay_body_event_handler(ngx_event_t *ev);
static ngx_int_t ngx_http_delay_body_init(ngx_conf_t *cf);
static ngx_http_module_t ngx_http_delay_body_module_ctx = {
NULL,
ngx_http_delay_body_init,
.......
};
ngx_module_t ngx_http_delay_body_filter_module = {
NGX_MODULE_V1,
&ngx_http_delay_body_module_ctx,
NULL,
NGX_HTTP_MODULE,
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
NGX_MODULE_V1_PADDING
};
static ngx_http_request_body_filter_pt ngx_http_next_request_body_filter;
static ngx_int_t
ngx_http_delay_body_filter(ngx_http_request_t *r, ngx_chain_t *in)
{
ngx_int_t rc;
ngx_chain_t *cl, *ln;
ngx_http_cleanup_t *cln;
ngx_http_delay_body_ctx_t *ctx;
ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
"delay request body filter");
ctx = ngx_http_get_module_ctx(r, ngx_http_delay_body_filter_module);
if (ctx == NULL) {
ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_delay_body_ctx_t));
ngx_http_set_ctx(r, ctx, ngx_http_delay_body_filter_module);
r->request_body->filter_need_buffering = 1;
}
if (ngx_chain_add_copy(r->pool, &ctx->out, in) != NGX_OK) {
return NGX_HTTP_INTERNAL_SERVER_ERROR;
}
if (!ctx->event.timedout) {
if (!ctx->event.timer_set) {
cln = ngx_http_cleanup_add(r, 0);
cln->handler = ngx_http_delay_body_cleanup;
cln->data = ctx;
ctx->event.handler = ngx_http_delay_body_event_handler;
ctx->event.data = r;
ctx->event.log = r->connection->log;
ngx_add_timer(&ctx->event, NGX_HTTP_DELAY_BODY);
}
return ngx_http_next_request_body_filter(r, NULL);
}
rc = ngx_http_next_request_body_filter(r, ctx->out);
for (cl = ctx->out; cl; ) {
ln = cl;
cl = cl->next;
ngx_free_chain(r->pool, ln);
}
ctx->out = NULL;
return rc;
}
static void
ngx_http_delay_body_cleanup(void *data)
{
ngx_http_delay_body_ctx_t *ctx = data;
if (ctx->event.timer_set) {
ngx_del_timer(&ctx->event);
}
}
static void
ngx_http_delay_body_event_handler(ngx_event_t *ev)
{
ngx_connection_t *c;
ngx_http_request_t *r;
r = ev->data;
c = r->connection;
ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0,
"delay request body event");
ngx_post_event(c->read, &ngx_posted_events);
}
static ngx_int_t
ngx_http_delay_body_init(ngx_conf_t *cf)
{
ngx_http_next_request_body_filter = ngx_http_top_request_body_filter;
ngx_http_top_request_body_filter = ngx_http_delay_body_filter;
return NGX_OK;
}
The following fields of the request determine how the request body is read:
- request_body_in_single_buf - Read the body to a single memory buffer.
- request_body_in_file_only - Always read the body to a file, even if fits in the memory buffer..
- request_body_no_buffering - Read the request body without buffering.
The request_body_no_buffering flag enables the unbuffered mode of reading a request body.
In this mode, after calling ngx_http_read_client_request_body(), the bufs chain might keep only a part of the body.
To read the next part, call the ngx_http_read_unbuffered_request_body(r) function.
The return value NGX_AGAIN and the request flag reading_body indicate that more data is available.
If bufs is NULL after calling this function, there is nothing to read at the moment. The request callback read_event_handler will be called when the next part of request body is available.
Response
In nginx an HTTP response is produced by sending the response header followed by the optional response body. Both header and body are passed through a chain of filters and eventually get written to the client socket.
An nginx module can install its handler into the header or body filter chain and process the output coming from the previous handler.
Response header
The ngx_http_send_header(r) function sends the output header.
Do not call this function until r->headers_out contains all of the data required to produce the HTTP response header.
static ngx_int_t
ngx_http_foo_content_handler(ngx_http_request_t *r)
{
ngx_int_t rc;
ngx_table_elt_t *h;
r->headers_out.status = NGX_HTTP_OK;
r->headers_out.content_length_n = 3;
h = ngx_list_push(&r->headers_out.headers);
h->hash = 1;
ngx_str_set(&h->key, "X-Foo");
ngx_str_set(&h->value, "foo");
rc = ngx_http_send_header(r);
...
}
Header filters
The ngx_http_send_header(r) function invokes the header filter chain by calling the first header filter handler stored in the ngx_http_top_header_filter variable. It's assumed that every header handler calls the next handler in the chain until the final handler ngx_http_header_filter(r) is called. The final header handler constructs the HTTP response based on r->headers_out and passes it to the ngx_http_writer_filter for output.
To add a handler to the header filter chain, store its address in the global variable ngx_http_top_header_filter at configuration time.
The following example of a header filter module adds the HTTP header "X-Foo: foo" to every response with status 200.
static ngx_int_t ngx_http_foo_header_filter(ngx_http_request_t *r);
static ngx_int_t ngx_http_foo_header_filter_init(ngx_conf_t *cf);
static ngx_http_module_t ngx_http_foo_header_filter_module_ctx = {
NULL,
ngx_http_foo_header_filter_init,
.......
};
ngx_module_t ngx_http_foo_header_filter_module = {
NGX_MODULE_V1,
&ngx_http_foo_header_filter_module_ctx,
NULL,
NGX_HTTP_MODULE,
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
NGX_MODULE_V1_PADDING
};
static ngx_http_output_header_filter_pt ngx_http_next_header_filter;
static ngx_int_t
ngx_http_foo_header_filter(ngx_http_request_t *r)
{
ngx_table_elt_t *h;
h = ngx_list_push(&r->headers_out.headers);
h->hash = 1;
ngx_str_set(&h->key, "X-Foo");
ngx_str_set(&h->value, "foo");
return ngx_http_next_header_filter(r);
}
static ngx_int_t
ngx_http_foo_header_filter_init(ngx_conf_t *cf)
{
ngx_http_next_header_filter = ngx_http_top_header_filter;
ngx_http_top_header_filter = ngx_http_foo_header_filter;
return NGX_OK;
}
Response body
To send the response body, call the ngx_http_output_filter(r, cl) function.
The following example produces a complete HTTP response with "foo" as its body.
For the example to work as subrequest as well as a main request, the last_in_chain flag is set in the last buffer of the output.
static ngx_int_t
ngx_http_bar_content_handler(ngx_http_request_t *r)
{
ngx_int_t rc;
ngx_buf_t *b;
ngx_chain_t out;
r->headers_out.status = NGX_HTTP_OK;
r->headers_out.content_length_n = 3;
rc = ngx_http_send_header(r);
b = ngx_calloc_buf(r->pool);
b->last_buf = (r == r->main) ? 1: 0;
b->last_in_chain = 1;
b->memory = 1;
b->pos = (u_char *) "foo";
b->last = b->pos + 3;
out.buf = b;
out.next = NULL;
return ngx_http_output_filter(r, &out);
}
Response body filters
The function ngx_http_output_filter(r, cl) invokes the body filter chain by calling the first body filter handler stored in the ngx_http_top_body_filter variable.
notes:链式反应，依次执行
It's assumed that every body handler calls the next handler in the chain until the final handler ngx_http_write_filter(r, cl) is called.
A body filter handler receives a chain of buffers.
Following is an example of a simple body filter that counts the number of bytes in the body.
The result is available as the $counter variable which can be used in the access log.
typedef struct {
off_t count;
} ngx_http_counter_filter_ctx_t;
static ngx_int_t ngx_http_counter_body_filter(ngx_http_request_t *r,
ngx_chain_t *in);
static ngx_int_t ngx_http_counter_variable(ngx_http_request_t *r,
ngx_http_variable_value_t *v, uintptr_t data);
static ngx_int_t ngx_http_counter_add_variables(ngx_conf_t *cf);
static ngx_int_t ngx_http_counter_filter_init(ngx_conf_t *cf);
static ngx_http_module_t ngx_http_counter_filter_module_ctx = {
ngx_http_counter_add_variables,
ngx_http_counter_filter_init,
.......
};
ngx_module_t ngx_http_counter_filter_module = {
NGX_MODULE_V1,
&ngx_http_counter_filter_module_ctx,
NULL,
NGX_HTTP_MODULE,
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
NGX_MODULE_V1_PADDING
};
static ngx_http_output_body_filter_pt ngx_http_next_body_filter;
static ngx_str_t ngx_http_counter_name = ngx_string("counter");
static ngx_int_t
ngx_http_counter_body_filter(ngx_http_request_t *r, ngx_chain_t *in)
{
ngx_chain_t *cl;
ngx_http_counter_filter_ctx_t *ctx;
ctx = ngx_http_get_module_ctx(r, ngx_http_counter_filter_module);
if (ctx == NULL) {
ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_counter_filter_ctx_t));
ngx_http_set_ctx(r, ctx, ngx_http_counter_filter_module);
}
for (cl = in; cl; cl = cl->next) {
ctx->count += ngx_buf_size(cl->buf);
}
return ngx_http_next_body_filter(r, in);
}
static ngx_int_t
ngx_http_counter_variable(ngx_http_request_t *r, ngx_http_variable_value_t *v,
uintptr_t data)
{
u_char *p;
ngx_http_counter_filter_ctx_t *ctx;
ctx = ngx_http_get_module_ctx(r, ngx_http_counter_filter_module);
p = ngx_pnalloc(r->pool, NGX_OFF_T_LEN);
v->data = p;
v->len = ngx_sprintf(p, "%O", ctx->count) - p;
v->valid = 1;
v->no_cacheable = 0;
v->not_found = 0;
return NGX_OK;
}
static ngx_int_t
ngx_http_counter_add_variables(ngx_conf_t *cf)
{
ngx_http_variable_t *var;
var = ngx_http_add_variable(cf, &ngx_http_counter_name, 0);
var->get_handler = ngx_http_counter_variable;
return NGX_OK;
}
static ngx_int_t
ngx_http_counter_filter_init(ngx_conf_t *cf)
{
ngx_http_next_body_filter = ngx_http_top_body_filter;
ngx_http_top_body_filter = ngx_http_counter_body_filter;
return NGX_OK;
}
Building filter modules
When writing a body or header filter, pay special attention to the filter's position in the filter order. There's a number of header and body filters registered by nginx standard modules.
The nginx standard modules register a number of head and body filters and it's important to register a new filter module in the right place with respect to them.
Normally, modules register filters in their postconfiguration handlers.
The order in which filters are called during processing is obviously the reverse of the order in which they are registered.
For third-party filter modules nginx provides a special slot HTTP_AUX_FILTER_MODULES.
To register a filter module in this slot, set the ngx_module_type variable to HTTP_AUX_FILTER in the module's configuration.
The following example shows a filter module config file assuming for a module with just one source file, ngx_http_foo_filter_module.c.
ngx_module_type=HTTP_AUX_FILTER ngx_module_name=ngx_http_foo_filter_module ngx_module_srcs="$ngx_addon_dir/ngx_http_foo_filter_module.c" . auto/module
Buffer reuse
When issuing or altering a stream of buffers, it's often desirable to reuse the allocated buffers.
A standard and widely adopted approach in nginx code is to keep two buffer chains for this purpose: free and busy.
The free chain keeps all free buffers, which can be reused.
The busy chain keeps all buffers sent by the current module that are still in use by some other filter handler.
Load balancing
The ngx_http_upstream_module provides the basic functionality needed to pass requests to remote servers.
Modules that implement specific protocols, such as HTTP or FastCGI, use this functionality.
The module also provides an interface for creating custom load-balancing modules and implements a default round-robin method.
The least_conn and hash modules implement alternative load-balancing methods, but are actually implemented as extensions of the upstream round-robin module and share a lot of code with it, such as the representation of a server group.
The keepalive module is an independent module that extends upstream functionality.
The ngx_http_upstream_module can be configured explicitly by placing the correspond upstream block into the configuration file, or implicitly by using directives such as proxy_pass that accept a URL that gets evaluated at some point into a list of servers.
The alternative load-balancing methods are available only with an explicit upstream configuration. The upstream module configuration has its own directive context NGX_HTTP_UPS_CONF.
The structure is defined as follows:
struct ngx_http_upstream_srv_conf_s {
ngx_http_upstream_peer_t peer;
void **srv_conf;
ngx_array_t *servers;
ngx_uint_t flags;
ngx_str_t host;
u_char *file_name;
ngx_uint_t line;
in_port_t port;
ngx_uint_t no_port;
#if (NGX_HTTP_UPSTREAM_ZONE)
ngx_shm_zone_t *shm_zone;
#endif
};
- srv_conf — Configuration context of upstream modules.
- servers — Array of ngx_http_upstream_server_t, the result of parsing a set of server directives in the upstream block.
- flags — Flags that mostly mark which features are supported by the load-balancing method. The features are configured as parameters of the server directive:
- host — Name of the upstream.
- file_name,line — Name of the configuration file and the line where the upstream block is located
- peer — object that holds generic methods for initializing upstream configuration:
A module that implements a load-balancing algorithm must set these methods and initialize private data.typedef struct { ngx_http_upstream_init_pt init_upstream; ngx_http_upstream_init_peer_pt init; void *data; } ngx_http_upstream_peer_t;- init_upstream(cf, us) — Configuration-time method responsible for initializing a group of servers and initializing the init() method in case of success.
- init(r, us) — Initializes a per-request ngx_http_upstream_peer_t.peer structure that is used for load balancing (not to be confused with the ngx_http_upstream_srv_conf_t.peer described above which is per-upstream).
It is passed as the data argument to all callbacks that deal with server selection.
When nginx has to pass a request to another host for processing, it uses the configured load-balancing method to obtain an address to connect to. The method is obtained from the ngx_http_upstream_t.peer object of type ngx_peer_connection_t:
struct ngx_peer_connection_s {
...
struct sockaddr *sockaddr;
socklen_t socklen;
ngx_str_t *name;
ngx_uint_t tries;
ngx_event_get_peer_pt get;
ngx_event_free_peer_pt free;
ngx_event_notify_peer_pt notify;
void *data;
#if (NGX_SSL || NGX_COMPAT)
ngx_event_set_peer_session_pt set_session;
ngx_event_save_peer_session_pt save_session;
#endif
...
};
The structure has the following fields:
- sockaddr, socklen, name — Address of the upstream server to connect to; this is the output parameter of a load-balancing method.
- data — The per-request data of a load-balancing method; keeps the state of the selection algorithm and usually includes the link to the upstream configuration. It is passed as an argument to all methods that deal with server selection (see below).
- tries — Allowed number of attempts to connect to an upstream server.
- get, free, notify, set_session, and save_session - Methods of the load-balancing module, described below.
All methods accept at least two arguments: a peer connection object pc and the data created by ngx_http_upstream_srv_conf_t.peer.init(). Note that it might differ from pc.data due to “chaining” of load-balancing modules.
- get(pc, data) — The method called when the upstream module is ready to pass a request to an upstream server and needs to know its address.
- NGX_DONE — the underlying connection was reused and there is no need to create a new connection to the upstream server. This value is set by the keepalive module.
- set_session(pc, data) and save_session(pc, data) — SSL-specific methods that enable caching sessions to upstream servers. The implementation is provided by the round-robin balancing method.
欢迎分享，转载请注明来源：内存溢出
微信扫一扫
支付宝扫一扫
评论列表（0条）