Django nginx gunicorn 502 ошибка

First. Don’t use if in an nginx conf. It’s bad. Like really, really horrible. Use the following instead:

location / {
    try_files $uri @proxy;
}

location @proxy {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://app_server_djangoapp;
}

See: http://wiki.nginx.org/IfIsEvil and http://wiki.nginx.org/Pitfalls

Now, as far as debugging goes. Your gunicorn workers are booting because there’s some fatal error. Try shutting down gunicorn. If you’re using supervisor:

sudo supervisorctl stop [gunicorn process name]

Then, from your project root run:

python manage.py run_gunicorn -c path/to/gunicorn.conf

Note any startup errors or if it actually boots, test your site in the browser. If you’re still not getting any meaningful info try just running the standard runserver

python manage.py runserver

Again, note any errors and if it loads fine, test your site in the browser. I suggest testing on localhost:8000 like you would in development. One of these should give you something to work with.

UPDATE

The error you’re getting says it can’t connect to «ind=127.0.0.1». Then, looking at the command you’re running, gunicorn_django -bind=127.0.0.1:8001, it’s easy to see the problem. You can specify the IP and port to bind to with either -b or --bind. Since you only used one - it’s interpreting the IP as ind=127.0.0.1, which is obviously not correct. You need to use:

gunicorn_django --bind=127.0.0.1:8001

Or

gunicorn_django -b 127.0.0.1:8001

Editor’s note: Gunicorn uses the term “master” to describe its primary process. Datadog does not use this term. Within this blog post, we will refer to this as “primary,” except for the sake of clarity in instances where we must reference a specific process name.

This post is part of a series on troubleshooting NGINX 502 Bad Gateway errors. If you’re not using Gunicorn, check out our other article on troubleshooting NGINX 502s with PHP-FPM as a backend.

Gunicorn is a popular application server for Python applications. It uses the Web Server Gateway Interface (WSGI), which defines how a web server communicates with and makes requests to a Python application. In production, Gunicorn is often deployed behind an NGINX web server. NGINX proxies web requests and passes them on to Gunicorn worker processes that execute the application.

NGINX will return a 502 Bad Gateway error if it can’t successfully proxy a request to Gunicorn or if Gunicorn fails to respond. In this post, we’ll examine some common causes of 502 errors in the NGINX/Gunicorn stack, and we’ll provide guidance on where you can find information you need to resolve these errors.

Some possible causes of 502s

In this section, we’ll describe how the following conditions can cause NGINX to return a 502 error:

• Gunicorn is not running
• NGINX can’t communicate with Gunicorn
• Gunicorn is timing out

If NGINX is unable to communicate with Gunicorn for any of these reasons, it will respond with a 502 error, noting this in its access log (/var/log/nginx/access.log) as shown in this example:

127.0.0.1 - - [08/Jan/2020:18:13:50 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.58.0"

NGINX’s access log doesn’t explain the cause of a 502 error, but you can consult its error log (/var/log/nginx/error.log) to learn more. For example, here is a corresponding entry in the NGINX error log that shows that the cause of the 502 error is that the socket doesn’t exist, possibly because Gunicorn isn’t running. (In the next section, we’ll look at how to detect and correct this problem.)

2020/01/08 18:13:50 [crit] 1078#1078: *189 connect() to unix:/home/ubuntu/myproject/myproject.sock failed (2: No such file or directory) while connecting to upstream, client: 127.0.0.1, server: localhost, request: "GET / HTTP/1.1", upstream: "http://unix:/home/ubuntu/myproject/myproject.sock:/", host: "localhost"

Gunicorn isn’t running

Note: This section includes a process name that uses the term “master.” Except when referring to specific processes, this article uses the term “primary” instead.

If Gunicorn isn’t running, NGINX will return a 502 error for any request meant to reach the Python application. If you’re seeing 502s, first check to confirm that Gunicorn is running. For example, on a Linux host, you can use a ps command like this one to look for running Gunicorn processes:

On a host where Gunicorn is serving a Flask app named myproject, the output of the above ps command would look like this:

ubuntu    3717  0.3  2.3  65104 23572 pts/0    S    15:45   0:00 gunicorn: master [myproject:app]
ubuntu    3720  0.0  2.0  78084 20576 pts/0    S    15:45   0:00 gunicorn: worker [myproject:app]

If the output of the ps command doesn’t show any Gunicorn primary or worker processes, see the documentation for guidance on starting your Gunicorn daemon.

In a production environment, you should consider using systemd to run your Python application as a service. This can make your app more reliable and scalable, since the Gunicorn daemon will automatically start serving your Python app when your server starts or when a new instance launches.

Once your Gunicorn project is configured as a service, you can use the following command to ensure that it starts automatically when your host comes up:

sudo systemctl enable myproject.service

Then you can use the list-unit-files command to see information about your service:

sudo systemctl list-unit-files | grep myproject

On a Linux server that has Gunicorn installed (even if it is not running), the output of this command will be:

myproject.service  enabled

To see information about your Gunicorn service, use this command:

sudo systemctl is-active myproject

This command should return an output of active. If it doesn’t, you can start the service with:

sudo service myproject start

If Gunicorn won’t start, it could be due to a typo in your unit file or your configuration file.
To find out why your application didn’t start, use the status command to see any errors that occurred on startup and use this information as a starting point for your troubleshooting:

sudo systemctl status myproject.service

NGINX can’t access the socket

When Gunicorn starts, it creates one or more TCP or Unix sockets to communicate with the NGINX web server. Gunicorn uses these sockets to listen for requests from NGINX.

To determine whether a 502 error was caused by a socket misconfiguration, confirm that Gunicorn and NGINX are configured to use the same socket. By default, Gunicorn creates a TCP socket located at 127.0.0.1:8000. You can override this default by using Gunicorn’s --bind switch to designate a different location—a different TCP socket, a Unix socket, or a file descriptor. The command shown here starts Gunicorn on the localhost using port 4999. (Flask apps typically run on port 5000, but that’s also the port used by the Datadog Agent by default, so we’ll adjust our examples to avoid any conflict.)

gunicorn --bind 127.0.0.1:4999 myproject:app

If your Gunicorn project is running as a systemd service, its unit file (e.g., /etc/systemd/system/myproject.service) will contain an ExecStart line where you can specify the bind information, similar to the command above. This is shown in the example unit file below:

[Unit]
Description=My Gunicorn project description
After=network.target

[Service]
User=ubuntu
Group=nginx
WorkingDirectory=/home/ubuntu/myproject
ExecStart=/usr/bin/gunicorn --bind 127.0.0.1:4999 myproject:app

[Install]
WantedBy=multi-user.target

Alternatively, your bind value can be in a Gunicorn configuration file. See the Gunicorn documentation for more information.

Next, check your nginx.conf file to ensure that the relevant location block specifies the same socket information Gunicorn is using. The example below contains an include directive that prompts NGINX to include proxy information in the headers of its requests, and a proxy_pass directive that specifies the same TCP socket named in the Gunicorn --bind options shown above.

location / {
  include proxy_params;
  proxy_pass http://127.0.0.1:4999;
}

If Gunicorn is listening on a Unix socket, the proxy_pass option will have a value in the form of /path/to/socket.sock, as shown below:

proxy_pass unix:/home/ubuntu/myproject/myproject.sock;

Just as with a TCP socket, you can prevent 502 errors by confirming that the path to this socket matches the one specified in the NGINX configuration.

Unix sockets are subject to Unix file system permissions. If you’re using a Unix socket, make sure its permissions allow read and write access by the group running NGINX. (You can use Gunicorn’s umask flag to designate the socket’s permissions.) If the permissions on the socket are incorrect, NGINX will log a 502 error in its access log, and a message like the one shown below in its error log:

2020/03/02 20:31:26 [crit] 18749#18749: *8551 connect() to unix:/home/ubuntu/myproject/myproject.sock failed (13: Permission denied) while connecting to upstream, client: 127.0.0.1, server: localhost, request: "GET / HTTP/1.1", upstream: "http://unix:/home/ubuntu/myproject/myproject.sock:/", host: "localhost"

Gunicorn is timing out

If your application is taking too long to respond, your users will experience a timeout error. Gunicorn’s timeout defaults to 30 seconds, and you can override this in the configuration file, on the command line, or in the systemd unit file. If Gunicorn’s timeout is less than NGINX’s timeout (which defaults to 60 seconds), NGINX will respond with a 502 error. The NGINX error log shown below indicates that the upstream process—which is Gunicorn—closed the connection before sending a valid response. In other words, this is the error log we see when Gunicorn times out:

2020/03/02 20:38:51 [error] 30533#30533: *17 upstream prematurely closed connection while reading response header from upstream, client: 127.0.0.1, server: localhost, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:4999/", host: "localhost"

Your Gunicorn log may also have a corresponding entry. (Gunicorn logs to stdout by default; see the documentation for information on configuring Gunicorn to log to a file.) The log line below is an example from a Gunicorn log, indicating that the application took too long to respond, and Gunicorn killed the worker thread:

[2020-03-02 18:15:05 +0000] [3417] [CRITICAL] WORKER TIMEOUT (pid:3438)

You can increase Gunicorn’s timeout value by adding the --timeout flag to the Gunicorn command you use to start your application—whether it’s in an ExecStart directive in your unit file, in a startup script, or using the command line as shown below:

gunicorn --timeout 60 myproject:app

Raising Gunicorn’s timeout could cause another issue: NGINX may time out before receiving a response from Gunicorn. The default NGINX timeout is 60 seconds; if you’ve raised your Gunicorn timeout above 60 seconds, NGINX will return a 504 Gateway Timeout error if Gunicorn hasn’t responded in time. You can prevent this by also raising your NGINX timeout. In the example below, we’ve raised the timeout value to 90 seconds by adding the fastcgi_read_timeout item to the http block in /etc/nginx/nginx.conf:

http { 
...
    fastcgi_buffers 8 16k;
    fastcgi_buffer_size 32k;
    fastcgi_connect_timeout 90s;
    fastcgi_send_timeout 90s;
    fastcgi_read_timeout 90s;
    proxy_read_timeout 90s;
}

Reload your NGINX configuration to apply this change:

Next, to determine why Gunicorn timed out, you can collect logs and application performance monitoring (APM) data that can reveal causes of latency within and outside your application.

Collect and analyze your logs

To troubleshoot 502 errors, you can collect your logs with a log management service. NGINX logging is active by default, and you can customize the location, format, and logging level.

By default, Gunicorn logs informational messages about server activity, including startup, shutdown, and the status of Gunicorn’s worker processes. You can add custom logging to your Python application code to collect logs corresponding to any notable events you want to track. This way, when you see a 502 error in NGINX’s access log, you can also reference the NGINX error log and your Python application logs. You can get even greater visibility by collecting logs from relevant technologies like caching servers and databases to correlate with any NGINX 502 error logs. Aggregating these logs in a single platform gives you visibility into your entire web stack, shortening your time spent troubleshooting and reducing your MTTR.

Collect APM data from your web stack

APM can help you identify bottlenecks and resolve issues—like 502 errors—that affect the performance of your app. The screenshot below shows a flame graph—a timeline of calls to all the services required to fulfill a request. Service calls are shown as horizontal spans, which illustrate the sequence of calls and the duration of each one.

Additionally, APM visualizations in Datadog show you your app’s error rates, request volume, and latency, giving you valuable context as you investigate performance problems like 502 errors.

Datadog’s Python tracing supports numerous Python frameworks, so it’s easy to start tracing your applications without making any changes to your code. See the Datadog docs for information on collecting APM data from your Python applications.

200 OK

The faster you can diagnose and resolve your application’s 502 errors, the better. Datadog allows you to analyze metrics, traces, logs, and network performance data from across your infrastructure. If you’re already a Datadog customer, you can start monitoring NGINX, Gunicorn, and more than
600 other technologies. If you don’t yet have a Datadog account, sign up for a 14-day free trial and get started in minutes.

Привет, проблема в том что пытаюсь настроить на vps веб сервер. настраивал по этому мануалу https://www.digitalocean.com/community/tutorials/h…

столкнулся с тем что nginx не общается с gunicorn
если запустить просто gunicorn —bind pystart.ru:8000 firstapp.wsgi
то все работает.

(13: Permission denied) while connecting to upstream, client:

drwxr-xr-x root root /
drwx—— root root root
drwxr-xr-x root root firstapp
srwxrwxrwx root www-data firstapp.sock

отчет запущенной службы
Sep 15 09:16:32 ovz1 systemd[1]: Starting A high performance web server and a reverse proxy server…
Sep 15 09:16:33 ovz1 systemd[1]: Failed to read PID from file /run/nginx.pid: Invalid argument
Sep 15 09:16:33 ovz1 systemd[1]: Started A high performance web server and a reverse proxy server.

Помогите найти решение проблемы. уже загуглил все что смог ничего не помогло.

Following this tutorial, I was able to set up Django, Gunicorn & nginx inside a virtualenv on an AWS EC2 instance (running Ubuntu 16.04), and then proceed to create an Upstart file to «daemonize» the entire thing.

After activating the virtualenv, I checked that:

1. Django works — I was able to access my Django project via port 8000 by running the following:

./manage.py runserver 0.0.0.0:8000

When visiting mydomain.com:8000 I was welcomed by the default Django page.

1. Gunicorn works & is able to serve the Django app — I was able to access my Django app by running this from my project’s folder (replace projectname with my actual project name):

gunicorn --bind 0.0.0.0:8000 projectname.wsgi:application

When visiting mydomain.com:8000 this time I was greeted again by the welcome page, and when appending /admin to the end of the URL I was able to see the login screen minus the CSS (because Gunicorn isn’t aware of the static CSS files, which is OK for now according to the tutorial).

From here I proceeded to create the following systemd file (when masterfolder is used to show the folder in which all the action takes place, myuser is my system user (ubuntu, as this is an Ubuntu instance) :

[Unit]
Description=gunicorn daemon
After=network.target

[Service]
User=myuser
Group=www-data
WorkingDirectory=/home/myuser/masterfolder
ExecStart=/home/myuser/masterfolder/myvirtualenv/bin/gunicorn --workers 3 --bind unix:/home/myuser/masterfolder/projectname.sock$

[Install]
WantedBy=multi-user.target`

I encountered two problems:

1. No socket is being created — when checking the nginx log files outside as well as inside the virtualenv (not entirely sure why they are the same, btw) I saw the same error:

2017/01/17 15:12:43 [crit] 12403#12403: *3 connect() to unix:/home/myuser/masterfolder/projectname.sock failed (2: No such file or directory) while connecting to upstream

Now, note that masterfolder is consistent with the folder hierarchy django-admin creates when I run a startproject:

masterfolder/
├── manage.py
└── projectname
├── __init__.py
├── settings.py
├── urls.py
└── wsgi.py

When looking inside masterfolder there is indeed no .sock file at all, never mind one that is named projectname.sock.

1. Therefore, despite the fact that nginx works, it throws a 502 —

This is how my etc/nginx/sites-availabe/projectname file looks like:

server {
    listen 80;
    server_name www.mydomain.com mydomain.com MYIPADDRESS;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        root /home/masterfolder/projectname;
    }

    location / {
        include proxy_params;
        proxy_pass http://unix:/home/myuser/mastefolder/projectname.sock;
    }
}

It’s symlinked correctly, of course, and after every change I make to try and fix it I run:

sudo systemctl daemon-reload
sudo systemctl start gunicorn
sudo systemctl enable gunicorn
sudo systemctl restart nginx

I feel as if I’m missing something basic here in the understanding of how all of this infrastructure works together. Feel free to point out errors in my understanding of the process, of course.

Answer by Linda Duncan

Jobs
Programming & related technical career opportunities

,

Questions

,

Stack Overflow
Public questions & answers

,Asking for help, clarification, or responding to other answers.

The solution was to add read and execute permissions to the root folder:

chmod o+rx /example_root_folder

Answer by Avi Haynes

I’m getting a 502 bad gateway on nginx, and the following on the logs: connect() to …myproject.sock failed (13: Permission denied) while connecting to upstream

I’m running wsgi and nginx on ubuntu, and I’ve been following [this guide from…,
Community Tools and Integrations
,I assume you followed the DigitalOcean tutorial “How To Serve Flask Applications with uWSGI and Nginx on Ubuntu 14.04”. It seems like few few people are running into the same issue. Take a look at this answer: http://stackoverflow.com/questions/29872174/wsgi-nginx-error-permission-denied-while-connecting-to-upstream,
Working on improving health and education, reducing inequality, and spurring economic growth? We’d like to help.

/myproject/myproject.ini

[uwsgi]
module = wsgi:app

master = true
processes = 5

socket = myproject.sock
chmod-socket = 666
vacuum = true

die-on-term = true

/etc/systemd/system/myproject.service

[Unit]
Description=uWSGI instance to serve myproject
After=network.target

[Service]
User=nickname
Group=www-data
WorkingDirectory=/home/nickname/myproject
Environment="PATH=/home/nickname/myprojectenv/bin"
ExecStart=/home/nickname/myprojectenv/bin/uwsgi --ini myproject.ini

[Install]
WantedBy=multi-user.target

/etc/nginx/sites-available/myproject

server {
    listen 80;
    server_name 163.172.172.76;

    location / {
        include uwsgi_params;
        uwsgi_pass unix:/home/nickname/myproject/myproject.sock;
    }
}

Answer by Alison Howe

If NGINX is unable to communicate with Gunicorn for any of these reasons, it will respond with a 502 error, noting this in its access log (/var/log/nginx/access.log) as shown in this example:,To troubleshoot 502 errors, you can collect your logs with a log management service. NGINX logging is active by default, and you can customize the location, format, and logging level.,In this section, we’ll describe how the following conditions can cause NGINX to return a 502 error:,NGINX’s access log doesn’t explain the cause of a 502 error, but you can consult its error log (/var/log/nginx/error.log) to learn more. For example, here is a corresponding entry in the NGINX error log that shows that the cause of the 502 error is that the socket doesn’t exist, possibly because Gunicorn isn’t running. (In the next section, we’ll look at how to detect and correct this problem.)

127.0.0.1 - - [08/Jan/2020:18:13:50 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.58.0"

Answer by Journey Russo

Asking for help, clarification, or responding to other answers.,I have tried looking for an answer, but mostly people get this error with uwsgi not Gunicorn. ,

Questions

,nginx error log has the following output:

nginx error log has the following output:

2019/09/20 17:23:20 [crit] 28847#28847: *2 connect() to unix:/home/ubuntu/app_test/app_test.sock failed (13: Permission denied) while connecting to upstream, client: <client-ip>, server: <server-ip>, request: "GET / HTTP/1.1", upstream: "http://unix:/home/ubuntu/app_test/app_test.sock:/", host: "<ip-address>"

I have the following configuration file for the app in nginx‘s sites-available directory with a simlink to in sites-enabled:

server {
    listen 80;
    server_name <server-ip>;

    location / {
        include proxy_params;
        proxy_pass http://unix:/home/ubuntu/app_test/app_test.sock;
    }
}

This is the service file in /etc/systemd/system/app_test.service

[Unit]
Description=Gunicorn instance to serve app_test
After=network.target

[Service]
User=ubuntu
Group=ubuntu
WorkingDirectory=/home/ubuntu/app_test
Environment="PATH=/home/ubuntu/app_test/appenv/bin"
ExecStart=/home/ubuntu/app_test/appenv/bin/gunicorn --workers 3 --bind unix:app_test.sock -m 002 wsgi:app

[Install]
WantedBy=multi-user.target

This is the app_test.pyfile:

from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "<h1 style='color:blue'>Hello There!</h1>"

if __name__ == "__main__":
    app.run(host='0.0.0.0')

Finally, this is wsgi.py:

from app_test import app

if __name__ == "__main__":
    app.run()

Answer by Otis Bonilla

Try to set the socket permission to 0666

– Alex Austin

Nov 20 ’14 at 20:10

,

How can a religion rationalize worshipping deities it has enslaved?

,I am having trouble running my application on a new DigitalOcean droplet. The machine runs CentOS 6.5,

The socket permission is set in the uwsgi.ini file. This happens each time the uwsgi application is started. Additionally, I have a working configuration of this same code on a different CentOS box and the socket permissions are the same.

– Brian Leach

Nov 20 ’14 at 21:04

I found my typo after many hours of searching. In /path/to/my/webapp/my_app_nginx.conf the line that reads

location @app {
        include uwsgi_params;
        uwsgi_pass unix:/home/webdev/mydevelopment/git/ers_portal_uwsgi.sock;
    }

should read

location @app {
        include uwsgi_params;
        uwsgi_pass unix:/home/webdev/mydevelopment/git/ers_portal/ers_portal_uwsgi.sock;
    }

Answer by Legend Higgins

We strongly recommend using Gunicorn behind a proxy server.,Then you can start your Gunicorn application using Gaffer:,Although there are many HTTP proxies available, we strongly advise that you
use Nginx. If you choose another proxy server you need to make sure that it
buffers slow clients when you use default Gunicorn workers. Without this
buffering Gunicorn will be easily susceptible to denial-of-service attacks.
You can use Hey to check if your proxy is behaving properly.,Then you can easily manage Gunicorn using Gaffer.

worker_processes 1;

user nobody nogroup;
# 'user nobody nobody;' for systems with 'nobody' as a group instead
error_log  /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
  worker_connections 1024; # increase if you have lots of clients
  accept_mutex off; # set to 'on' if nginx worker_processes > 1
  # 'use epoll;' to enable for Linux 2.6+
  # 'use kqueue;' to enable for FreeBSD, OSX
}

http {
  include mime.types;
  # fallback in case we can't determine a type
  default_type application/octet-stream;
  access_log /var/log/nginx/access.log combined;
  sendfile on;

  upstream app_server {
    # fail_timeout=0 means we always retry an upstream even if it failed
    # to return a good HTTP response

    # for UNIX domain socket setups
    server unix:/tmp/gunicorn.sock fail_timeout=0;

    # for a TCP configuration
    # server 192.168.0.7:8000 fail_timeout=0;
  }

  server {
    # if no Host match, close the connection to prevent host spoofing
    listen 80 default_server;
    return 444;
  }

  server {
    # use 'listen 80 deferred;' for Linux
    # use 'listen 80 accept_filter=httpready;' for FreeBSD
    listen 80;
    client_max_body_size 4G;

    # set the correct host(s) for your site
    server_name example.com www.example.com;

    keepalive_timeout 5;

    # path for static files
    root /path/to/app/current/public;

    location / {
      # checks for static file, if not found proxy to app
      try_files $uri @proxy_to_app;
    }

    location @proxy_to_app {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header Host $http_host;
      # we don't want nginx trying to do something clever with
      # redirects, we set the Host: header above already.
      proxy_redirect off;
      proxy_pass http://app_server;
    }

    error_page 500 502 503 504 /500.html;
    location = /500.html {
      root /path/to/app/current/public;
    }
  }
}

Answer by Jackson Beck

#Check if the gunicorn has write access to the folder you want to add 
#the file. If it doesn't have, then use
chmod -R 777 the_folder_you_want_to_write_to

Answer by Leila Lara

I am getting this error in my nginx-error.log file:,The browser also shows a 502 Bad Gateway Error. The output of a curl is the same, Bad Gateway html,Here is my nginx.conf file:,Make sure there are no security implications for your use-case before running this.

I am getting this error in my nginx-error.log file:

2014/02/17 03:42:20 [crit] 5455#0: *1 connect() to unix:/tmp/uwsgi.sock failed (13: Permission denied) while connecting to upstream, client: xx.xx.x.xxx, server: localhost, request: "GET /users HTTP/1.1", upstream: "uwsgi://unix:/tmp/uwsgi.sock:", host: "EC2.amazonaws.com"

nginx.conf

worker_processes 1;
worker_rlimit_nofile 8192;

events {
  worker_connections  3000; 
}

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on; 
    #tcp_nopush     on; 

    keepalive_timeout  65; 

    #gzip  on; 

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

I believe that ansible-playbook figured out my uwsgi configuration since when I run this command

uwsgi -s /tmp/uwsgi.sock -w my_app:app

it starts up and outputs this:

*** Starting uWSGI 2.0.1 (64bit) on [Mon Feb 17 20:03:08 2014] ***
compiled with version: 4.7.3 on 10 February 2014 18:26:16
os: Linux-3.11.0-15-generic #25-Ubuntu SMP Thu Jan 30 17:22:01 UTC 2014
nodename: ip-10-9-xxx-xxx
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /home/username/Project
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 4548
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 3
Python version: 2.7.5+ (default, Sep 19 2013, 13:52:09)  [GCC 4.8.1]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1f60260
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72760 bytes (71 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x1f60260 pid: 26790 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 26790, cores: 1)

Answer by Lainey Hughes

When you hit the URL with a web browser, you probably get a 502 Bad Gateway error. This could be standard permissions issues on the UNIX socket, or more complex access control problems with SELinux.,Once you’ve confirmed you get the page and there’s no other permissions issues, re-enable SELinux.,Here are some sample unit files for your consideration. First, we’ll set up the socket.,In the nginx error log you may see the following.

I want to use the latest version of Django, so I’ll need a Python 3.5 or later virtual environment (virtualenv). Reading the Beta notes, I can see that Python 3.6 is available so let’s check out the system.

[[email protected] ~]$ python
-bash: python: command not found
[[email protected] ~]$ python3
-bash: python3: command not found

OK then, to get Python I only need to install two packages, python3-pip will get pulled in as a dependency.

sudo yum install python36 python3-virtualenv

I know I want to use the default versions of PostgreSQL and Nginx available in RHEL 8, so I can install those with Yum.

sudo yum install nginx postgresql-server

That fits the bill so we’ll put everything we need in a directory under /srv that our application user (cloud-user) owns.

sudo mkdir /srv/djangoapp
sudo chown cloud-user:cloud-user /srv/djangoapp
cd /srv/djangoapp
virtualenv django
source django/bin/activate
pip3 install django gunicorn psycopg2
./django-admin startproject djangoapp /srv/djangoapp

Setting up PostgreSQL and Django is straightforward: create the database, create the user, and set up permissions. One thing to note during the initial setup of PostgreSQL is the postgresql-setup script shipped with the postgresql-server package. This script can help with basic database cluster administration tasks, like initialization or upgrades. For setting up a new PostgreSQL instance on a RHEL system, we’ll run:

sudo /usr/bin/postgresql-setup --initdb

Then we can start PostgreSQL with systemd, create the database, and set up the project in Django. Remember to restart PostgreSQL after you make changes to the client authentication configuration file (usually pg_hba.conf) to set up host password authentication for the application user. I spent far too much time chasing down that problem. If you have other issues, make sure you changed the IPv4 and IPv6 entries in the pg_hba.conf.

systemctl enable --now postgresql

sudo -u postgres psql
postgres=# create database djangoapp;
postgres=# create user djangouser with password 'qwer4321';
postgres=# alter role djangouser set client_encoding to 'utf8';
postgres=# alter role djangouser set default_transaction_isolation to 'read committed';
postgres=# alter role djangouser set timezone to 'utc';
postgres=# grant all on DATABASE djangoapp to djangouser;
postgres=# q

In /var/lib/pgsql/data/pg_hba.conf:

# IPv4 local connections:
host    all        all 0.0.0.0/0                md5
# IPv6 local connections:
host    all        all ::1/128                 md5

In /srv/djangoapp/settings.py:

# Database
DATABASES = {
   'default': {
       'ENGINE': 'django.db.backends.postgresql_psycopg2',
       'NAME': '{{ db_name }}',
       'USER': '{{ db_user }}',
       'PASSWORD': '{{ db_password }}',
       'HOST': '{{ db_host }}',
   }
}

Once you’ve got the project settings.py configured and the database configured, you can run the development server to check your work. Creating an admin user after you’ve started the development server is a good way to test if the database connection is working.

./manage.py runserver 0.0.0.0:8000
./manage.py createsuperuser

Here are some sample unit files for your consideration. First, we’ll set up the socket.

[Unit]
Description=Gunicorn WSGI socket

[Socket]
ListenStream=/run/gunicorn.sock

[Install]
WantedBy=sockets.target

Next, we’ll set up the Gunicorn daemon.

[Unit]
Description=Gunicorn daemon
Requires=gunicorn.socket
After=network.target

[Service]
User=cloud-user
Group=cloud-user
WorkingDirectory=/srv/djangoapp

ExecStart=/srv/djangoapp/django/bin/gunicorn 
         --access-logfile - 
         --workers 3 
         --bind unix:gunicorn.sock djangoapp.wsgi

[Install]
WantedBy=multi-user.target

For Nginx, it’s just a matter of creating the proxy configs and setting the static content directory if you made one. In RHEL, the config files for Nginx live in /etc/nginx/conf.d. You can drop the example in as /etc/nginx/conf.d/default.conf, and start the service. Be sure to set the server_name to what matches your host.

server {
   listen 80;
   server_name 8beta1.example.com;

   location = /favicon.ico { access_log off; log_not_found off; }
   location /static/ {
       root /srv/djangoapp;
   }

   location / {
       proxy_set_header Host $http_host;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_set_header X-Forwarded-Proto $scheme;
       proxy_pass http://unix:/run/gunicorn.sock;
   }
}

In the nginx error log you may see the following.

2018/12/18 15:38:03 [crit] 12734#0: *3 connect() to unix:/run/gunicorn.sock failed (13: Permission denied) while connecting to upstream, client: 192.168.122.1, server: 8beta1.example.com, request: "GET / HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/", host: "8beta1.example.com"

If we test Gunicorn directly, we get an empty reply.

curl --unix-socket /run/gunicorn.sock 8beta1.example.com

Why?  If you look at the journal, SELinux is probably the culprit. Since we’re running a daemon that doesn’t have a policy, it gets labeled as init_t. So let’s test our theory.

sudo setenforce 0

Once you’ve confirmed you get the page and there’s no other permissions issues, re-enable SELinux.

sudo setenforce 1

To create a specific permissive domain for Gunicorn, we’ll need a policy and to label some files to match. We also need the tools to make compile new policies.

sudo yum install selinux-policy-devel

gunicorn.te:

policy_module(gunicorn, 1.0)

type gunicorn_t;
type gunicorn_exec_t;
init_daemon_domain(gunicorn_t, gunicorn_exec_t)
permissive gunicorn_t;

We can compile this policy file and add it to our system.

make -f /usr/share/selinux/devel/Makefile
sudo semodule -i gunicorn.pp

sudo semanage permissive -a gunicorn_t
sudo semodule -l | grep permissive

Let’s take a look at anything else SELinux may be blocking, other than everything our unknown daemon wants to touch.

sudo ausearch -m AVC

type=AVC msg=audit(1545315977.237:1273): avc:  denied { write } for pid=19400 comm="nginx" name="gunicorn.sock" dev="tmpfs" ino=52977 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file permissive=0

SELinux is stopping Nginx from writing to the UNIX socket that Gunicorn uses. Normally we’d start adjusting policies, but we know there’s more work to do. We can also set an existing enforcing domain to be a permissive domain. So let’s move httpd_t to permissive as well. That gives Nginx the access it needs but we can continue working and troubleshooting.

sudo semanage permissive -a httpd_t

OK, now that SELinux is enforcing (really, you don’t want to ship this configured with SELinux in permissive mode) and our permissive domains loaded, we need to figure out what we need to label as gunicorn_exec_t to get everything working again. Hit the website to create more denials.

sudo ausearch -m AVC -c gunicorn

But there’s also this message:

type=AVC msg=audit(1545320700.070:1542): avc:  denied { execute } for pid=20704 comm="(gunicorn)" name="python3.6" dev="vda3" ino=8515706 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file permissive=0

If we look at the status of the gunicorn service or check ps, we don’t have any running processes. It looks like gunicorn is trying to call the Python interpreter in our virtualenv, perhaps to start up workers. So for now, let’s label these two binaries and see if we get our Django test page.

chcon -t gunicorn_exec_t /srv/djangoapp/django/bin/gunicorn /srv/djangoapp/django/bin/python3.6

You’ll need to restart the gunicorn service to pick up the new label. You can either restart it directly or stop the service and let the socket start it when you hit the website with a browser. Check the processes got the right labels with ps.

ps -efZ | grep gunicorn

If you pull up the AVCs that get logged now, you should see that the last item says permissive=1 for anything related to our application, and permissive=0 for the rest of the system. We can find a better way to fix any issues once we understand all of the real access our app might need. But until then, the system is better protected and we get usable auditing for our Django project.

sudo ausearch -m AVC