Para Debian bullseye 11

Instalación de requisitos

apt update
apt upgrade
sudo apt install apt-transport-https openjdk-11-jre-headless uuid-runtime pwgen dirmngr gnupg wget git nginx-full certbot python3-certbot-nginx

Instalación de mongodb

wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | sudo apt-key add -
echo "deb http://repo.mongodb.org/apt/debian buster/mongodb-org/4.2 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.2.list
 
apt-get update
sudo apt-get install -y mongodb-org
 
systemctl daemon-reload
systemctl enable mongod.service
systemctl restart mongod.service
systemctl --type=service --state=active | grep mongod

Instalación de Elasticsearch

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/oss-7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
apt update && sudo apt install elasticsearch-oss
apt update && sudo apt install elasticsearch-oss

Configuración de Elasticsearch

cat /etc/elasticsearch/elasticsearch.yml
tee -a /etc/elasticsearch/elasticsearch.yml > /dev/null << EOT
cluster.name: graylog
action.auto_create_index: false
EOT
 
cat  /etc/elasticsearch/elasticsearch.yml
 
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl restart elasticsearch.service
systemctl restart elasticsearch.service

Instalación de graylog

wget https://packages.graylog2.org/repo/packages/graylog-4.2-repository_latest.deb
dpkg -i graylog-4.2-repository_latest.deb
apt-get update && sudo apt-get install graylog-server graylog-enterprise-plugins graylog-integrations-plugins graylog-enterprise-integrations-plugins
 
systemctl enable graylog-server.service
systemctl start graylog-server.service
 
apt remove graylog-enterprise-plugins graylog-enterprise-integrations-plugins

Creamos la carpeta donde vamos a guardar los certificados

mkdir certificados
cd certificados

Creamos la llave para nuestro CA

certtool --generate-privkey --outfile ca-key.pem

Creamos el certificado CA con las siguientes respuestas:

Generating a self signed certificate...
Please enter the details of the certificate's distinguished name. Just press enter to ignore a field.
Common name: auth.ejemplo.com
UID: 
Organizational unit name: authority 
Organization name: ejemplo
Locality name: San Salvador
State or province name: San Salvador
Country name (2 chars): SV
Enter the subject's domain component (DC): auth.ejemplo.com
Enter an additional domain component (DC): 
This field should not be used in new certificates.
E-mail: 
Enter the certificate's serial number in decimal (123) or hex (0xabcd)
(default is 0x3efb2af4747406ca4a4d6e8b264f651c0b41fd4e)
value: 


Activation/Expiration time.
The certificate will expire in (days): 3651


Extensions.
Does the certificate belong to an authority? (y/N): y
Path length constraint (decimal, -1 for no constraint): 
Is this a TLS web client certificate? (y/N): 
Will the certificate be used for IPsec IKE operations? (y/N): 
Is this a TLS web server certificate? (y/N): 
Enter a dnsName of the subject of the certificate: auth.ejemplo.com
Enter an additional dnsName of the subject of the certificate: 
Enter a URI of the subject of the certificate: 
Enter the IP address of the subject of the certificate: 
Enter the e-mail of the subject of the certificate: 
Will the certificate be used for signing (required for TLS)? (Y/n): y
Will the certificate be used for data encryption? (y/N): 
Will the certificate be used to sign OCSP requests? (y/N): 
Will the certificate be used to sign code? (y/N): 
Will the certificate be used for time stamping? (y/N): 
Will the certificate be used for email protection? (y/N): 
Will the certificate be used to sign other certificates? (Y/n): y
Will the certificate be used to sign CRLs? (y/N): 
Enter the URI of the CRL distribution point: 

certtool --generate-self-signed --load-privkey ca-key.pem --outfile ca.pem

Cambiamos permisos a la llave Importante no copiar este archivo que es la que nos da la seguridad.

chmod 400 ca-key.pem
ls -alh

Creamos la llave para los clientes

certtool --generate-privkey --outfile key.pem --bits 2048

Creamos la solicitud de certificado usando el CA para los clientes con estas respuestas:

** Note: You may use '--sec-param Medium' instead of '--bits 2048'
Generating a 2048 bit RSA private key...
root@logs:~/certificados/new# certtool --generate-request --load-privkey key.pem --outfile request.pem
Generating a PKCS #10 certificate request...
Common name: client.ejemplo.com
Organizational unit name: client
Organization name: ejemplo
Locality name: San Salvador
State or province name: San Salvador
Country name (2 chars): SV
Enter the subject's domain component (DC): client.ejemplo.com
Enter an additional domain component (DC): 
UID: 
Enter a dnsName of the subject of the certificate: client.ejemplo.com
Enter an additional dnsName of the subject of the certificate: 
Enter a URI of the subject of the certificate: 
Enter the IP address of the subject of the certificate: 
Enter the e-mail of the subject of the certificate: 
Enter a challenge password: 
Does the certificate belong to an authority? (y/N): n
Will the certificate be used for signing (DHE ciphersuites)? (Y/n): 
Will the certificate be used for encryption (RSA ciphersuites)? (Y/n): 
Will the certificate be used to sign code? (y/N): 
Will the certificate be used for time stamping? (y/N): 
Will the certificate be used for email protection? (y/N): 
Will the certificate be used for IPsec IKE operations? (y/N): 
Will the certificate be used to sign OCSP requests? (y/N): 
Is this a TLS web client certificate? (y/N): y
Is this a TLS web server certificate? (y/N): y

certtool --generate-request --load-privkey key.pem --outfile request.pem
ls -alh

Creamos el certificado para los clientes con estas respuestas:

Generating a signed certificate...
Enter the certificate's serial number in decimal (123) or hex (0xabcd)
(default is 0x29a9678b6734082b227281cc331037bfb1cac595)
value: 


Activation/Expiration time.
The certificate will expire in (days): 3650


Extensions.
Do you want to honour all the extensions from the request? (y/N): 
Does the certificate belong to an authority? (y/N): n
Is this a TLS web client certificate? (y/N): y
Will the certificate be used for IPsec IKE operations? (y/N): 
Is this a TLS web server certificate? (y/N): y
Enter a dnsName of the subject of the certificate: client.ejemplo.com
Enter an additional dnsName of the subject of the certificate: 
Enter a URI of the subject of the certificate: 
Enter the IP address of the subject of the certificate: 
Will the certificate be used for signing (DHE ciphersuites)? (Y/n): 
Will the certificate be used for encryption (RSA ciphersuites)? (Y/n): 
Will the certificate be used for data encryption? (y/N): 
Will the certificate be used to sign OCSP requests? (y/N): 
Will the certificate be used to sign code? (y/N): 
Will the certificate be used for time stamping? (y/N): 
Will the certificate be used for email protection? (y/N): 

certtool --generate-certificate --load-request request.pem --outfile cert.pem --load-ca-certificate ca.pem --load-ca-privkey ca-key.pem

Ahora borramos la solicitud de certificado

rm -f request.pem

Creamos la carpeta para los clientes

mkdir certificados-cliente
cp ca.pem  cert.pem  key.pem certificados-cliente/

Estas es la carpeta que vamos a copiar a nuestros clientes

Configuración de Graylog

echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1
 
nano /etc/graylog/server/server.conf
 
cat /etc/graylog/server/server.conf |grep -P ""^[^#]  
is_master = true
node_id_file = /etc/graylog/server/node-id
password_secret = unadelascadenasmasgrandes
root_password_sha2 = lacadenaquesacamosenelcomandoanterior
bin_dir = /usr/share/graylog-server/bin
data_dir = /var/lib/graylog-server
plugin_dir = /usr/share/graylog-server/plugin
http_publish_uri = http://localhost:9000/
http_external_uri =https://logs.ejemplo.com/graylog/
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://localhost/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
proxied_requests_thread_pool_size = 32

cd /etc/nginx/sites-enabled/
rm default

Creamos la configuración para tener el graylog en un subfolder del sitio

cat  << EOF |tee -a /etc/nginx/sites-available/logs.ejemplo.com.conf
server {
    listen 80;
    listen [::]:80;
    server_name logs.ejempplo.com;
    # enforce https
    return 301 https://$server_name:443$request_uri;
}
 
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name logs.ejemplo.com;
 
    root /var/www/html;
 
    # Add index.php to the list if you are using PHP
    index index.html index.htm index.nginx-debian.html;
 
    ssl_certificate     /etc/letsencrypt/live/logs.ejemplo.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/logs.ejemplo.com/privkey.pem;
    #ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
 
    location / {
            # First attempt to serve request as file, then
            # as directory, then fall back to displaying a 404.
            try_files $uri $uri/ =404;
    }
 
    location /graylog/ {
      proxy_set_header Host $http_host;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Server $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Graylog-Server-URL https://$server_name/graylog/;
      rewrite ^/graylog/(.*)$ /$1 break;
 
      proxy_pass       http://127.0.0.1:9000;
    }
 
 
  access_log            /var/log/nginx/logs.ejemplo.com-access.log combined;
  error_log             /var/log/nginx/logs.ejemplo.com-error.log warn;
}
 
EOF

Reiniciamos el nginx para que lea la configuración

systemctl restart nginx

Ahora accedemos al servidor con: https://logs.ejemplo.com/graylog/

Usuario: admin

Clave: la clave que definieron anteriormente

Ahora ya logueados en el graylog, agregamos un INPUT, desde: System→Inputs Seleccionamos Syslog TCP→Lanch new Input Configurando los siguientes valores:

Title: Servidores
Bind Address: 0.0.0.0
Port: 8514
TLS Cert File (Optional): /etc/certificados/ca.pem
TLS Private key file (Optional): /etc/certificados/ca-key.pem
Enable TLS: true

Luego iniciar el Input

Y verificar el puerto con el comando

ss  -putona|grep 8514

Instalamos el soporte para certificados TLS

apt install -y rsyslog-gnutls

Copiamos los certificados

cp -rp ../certificados /etc/

Agregamos la configuración el rsyslog para que envie los logs al servidor central

cat  >> /etc/rsyslog.conf << EOL
 
#Para el servidor de logs
\$DefaultNetstreamDriver gtls
#$DefaultNetstreamDriver ossl
 
\$DefaultNetstreamDriverCAFile   /etc/certificados/ca.pem
\$DefaultNetstreamDriverCertFile /etc/certificados/cert.pem
\$DefaultNetstreamDriverKeyFile  /etc/certificados/key.pem
 
\$ActionSendStreamDriverAuthMode x509/name
\$ActionSendStreamDriverPermittedPeer auth.ejemplo.com
\$ActionSendStreamDriverMode 1 # run driver in TLS-only mode
 
# Increase the amount of open files rsyslog is allowed, which includes open tcp sockets
# This is important if there are many clients.
# http://www.rsyslog.com/doc/rsconf1_maxopenfiles.html
#$MaxOpenFiles 2048
 
*.*@@logs.ejemplo.com:8514;RSYSLOG_SyslogProtocol23Format
EOL

Reiniciamos y verificamos que no haya problemas

systemctl restart rsyslog
 
tail -n 50 /var/log/syslog

Para hacer que nginx envie los logs al syslog se debe agregar en la configuración

    error_log syslog:server=unix:/dev/log,tag=nginx,severity=error;
    access_log syslog:server=unix:/dev/log,tag=nginx,severity=info combined;

ó Para enviar los logs del servidor web es de agregar en el archivo /etc/rsyslog.d/20-nginx.conf:

module(load="imfile" PollingInterval="10")
 
#Para el servidor de logs
$DefaultNetstreamDriver gtls
#$DefaultNetstreamDriver ossl
 
$DefaultNetstreamDriverCAFile   /etc/certificados/ca.pem
$DefaultNetstreamDriverCertFile /etc/certificados/cert.pem
$DefaultNetstreamDriverKeyFile  /etc/certificados/key.pem
 
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer auth.floresvillatoro.com
$ActionSendStreamDriverMode 1 # run driver in TLS-only mode
 
#local7.info @@logs.floresvillatoro.com:8514;RSYSLOG_SyslogProtocol23Format
 
 
 
$InputFileName  /var/log/nginx/*access.log
$InputFileTag nginx:
$InputFileSeverity error
$InputRunFileMonitor
$InputFileFacility local6
 
local6.* @@logs.floresvillatoro.com:8514;RSYSLOG_SyslogProtocol23Format
 
$InputFileName  /var/log/nginx/*access.log
$InputFileTag nginx:
$InputFileSeverity info
$InputRunFileMonitor
$InputFileFacility local7
 
local7.* @@logs.floresvillatoro.com:8514;RSYSLOG_SyslogProtocol23Format

Para ello necesitan agregar en el archivo

transport_email_enabled = true
transport_email_protocol = smtp
transport_email_hostname = smtp.gmail.com
transport_email_port = 465
transport_email_use_auth = true
transport_email_use_tls = false
transport_email_use_ssl = true
transport_email_auth_username = MICUENTAEN@gmail.com
transport_email_auth_password = MISUPERCLAVE
transport_email_subject_prefix = [graylog]
transport_email_from_email = MICUENTAEN@gmail.com
 
transport_email_web_interface_url = https://GRAYLOGWWW/graylog/

Reiniciar el graylog

systemctl  restart graylog-server.service 

Crear la notificación via web y sino funciona revisar:

1. Que se le haya dado permisos a esa aplicación de hacer login https://g.co/allowaccess
2. Haber habilitado las aplicaciones menos seguras https://www.google.com/settings/security/lesssecureapps
3. Resolver el captcha de autenticacion https://accounts.google.com/DisplayUnlockCaptcha

Revisar este enlace para mas información: * https://serverfault.com/questions/635139/how-to-fix-send-mail-authorization-failed-534-5-7-14

1. Crear una cuenta en https://dev.maxmind.com/geoip/geolite2-free-geolocation-data?lang=en y descargar la base GeoLite2-City_XXXXXXXX.tar.gz, descomprimirlo y poner el archivo en /etc/graylog/server/GeoLite2-City.mmdb.

rule "GeoIP lookup: src_ip"
 
 
when
 
  has_field("facility")
 
then
//debug("HELLOOO1");
 
let srcc_ip = to_string(regex("([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})", to_string($message.message) ));
//debug("HELLOOO");
//debug(to_string($message.message));
 
 
//debug(substring(srcc_ip,3,(length(srcc_ip)-to_long("1") )));
//debug("HELLOOOend");
let srcc_ip=to_string(substring(srcc_ip,3,(length(srcc_ip)-to_long("1"))));
 
let geo = lookup("geoip", to_string(srcc_ip));
 
set_field("src_ip", to_string(srcc_ip));
 
 
set_field("src_ip_geo_location", geo["coordinates"]);
 
set_field("src_ip_geo_country", geo["country"].iso_code);
 
set_field("src_ip_geo_city", geo["city"].names.en);
 
end

rule "GeoIP lookup: src_ip"
 
when
 
  has_field("src_ip")
 
then
 
let geo = lookup("geoip", to_string($message.src_ip)));
 
set_field("src_ip_geo_location", geo[“coordinates”]);
 
set_field("src_ip_geo_country", geo[“country”].iso_code);
 
set_field("src_ip_geo_city", geo[“city”].names.en);
 
end

1. Verificar que el campi de IP (puede ser src_ip) este en los campos a obtener, sino puede sacarlos con reglas groks en Input→ELINPUT→Manage Extractors→Agregar regla o directamente desde la regla pipeline(mas complicado).
2. Usar la funcion DEBUG para imprimir mensajes desde la regla pipeline en caso de que no agregue los campos de geolocalizacion.
3. Si los datos de geolocalizacion están en el registro, verificar que:
4. - El mapa tenga agrupacion por src_ip_geolocation y metrica Count(src_ip_geolocation)
5. - Que el procesado GeoIP Resolver este al final de la lista en System→Configuration→Message Processors Configuration

• https://www.tecmint.com/install-rsyslog-centralized-logging-in-centos-ubuntu/
• https://www.rsyslog.com/doc/v8-stable/tutorials/database.html
• https://wiki.debian.org/Rsyslog
• https://www.caktusgroup.com/blog/2019/06/20/how-to-set-up-centralized-log-server-rsyslog/
• https://www.thegeekdiary.com/how-to-configure-rsyslog-server-to-accept-logs-via-ssl-tls/
• https://rsyslog.readthedocs.io/en/latest/tutorials/tls.html
• https://www.rsyslog.com/doc/v8-stable/tutorials/tls.html
• https://www.rsyslog.com/files/temp/doc-indent/configuration/modules/omfwd.html
• https://devconnected.com/the-definitive-guide-to-centralized-logging-with-syslog-on-linux/
• https://www.graylog.org/
• https://docs.graylog.org/v1/docs/debian
• https://computingforgeeks.com/install-graylog-on-debian-with-lets-encrypt-ssl/
• https://docs.graylog.org/v1/docs/syslog
• https://community.graylog.org/t/how-to-send-email-alerts-using-gmail/4103
• https://www.tecmint.com/configure-custom-access-and-error-log-formats-in-nginx/
• https://luvpreetsingh.github.io/nginx-to-rsyslog/
• https://www.linode.com/docs/guides/how-to-install-and-configure-graylog2-on-debian-9/
• https://www.graylog.org/post/how-to-set-up-graylog-geoip-configuration
• https://community.graylog.org/t/regex-pipeline-get-ip-from-string-field/2294/6
• https://archivedocs.graylog.org/en/4.2/pages/geolocation.html
• https://community.graylog.org/t/geo-ip-not-working/14846/7
• https://docs.graylog.org/docs/extractors
• https://www.stackhero.io/en/services/Graylog/documentations/Choose-inputs-types#raw-plaintext-format