I have a very simple setup running Gitea which I love. However, I enabled Elastic Search because it makes searching much faster than the default method.
I have a VPS running 16GB memory. The only things running on it are Nginx, PHP, Mysql, docker, and a few other things. Very rarely I ever hit over 6GB usage.
The issue comes when I enable Elastic Search. It seems to wipe me out at 15.7GB usage out of 16GB as soon as I start it up.
I searched online and found out about the /etc/elasticsearch/jvm.options.d/jvm.options and adding
-XmxXG
-XmsXG
The question is, what should this amount be. I read that by default, Elastic uses 50%, however, when I started it up, it was wiping me out of memory and making the system almost have a stroke.
But setting it to 2GB seems to make it not as responsive on the Gitea website, sometimes even timing the website out.
So I'm not sure what "range" I should be using here. Or if I'm going to have to upgrade my VPS to 32GB in order to run this properly.
Reverse proxy is exactly why I don't have more things setup in docker. I haven't quite figured out how it, nginx, and the app work together yet.
I had to setup caddy when I installed vaultwarden, and while that was easy because I had a very good guide to assist me, I would have been completely and totally lost if I had to setup caddy2 on my own.
So I definitely need to sit down one day and just do a full day's read on reverse proxy, how it works with Docker and its function, and what I can do with it. Because the vaultwarden setup made it no easier to understand.
I wanted to actually move nginx and mysql over to docker, but reverse proxy is also the reason that's holding me back.
If you already have Caddy running on that same Docker host then its very simple to add another proxy target to that through the Caddyfile.
I have been using Traefik for years and i am mostly happy with it but recently spend a day on trying out Caddy together with Authelia for authentication. Here is what came out of it as a very basic example to use them together. Its using a custom Docker image for Caddy that contains a few extra modules but it can easily be replaced with the basic official one, depending on what modules you need (for example, Lets Encrypt with dns01-challenge requires modules for DNS providers). My example uses www.desec.io for that but Cloudflare, DuckDNS etc. are possible too.
docker-compose.yml
version: "3.3"
networks:
caddy:
external: true
services:
caddy:
container_name: caddy
image: l33tlamer/caddy-desec:latest
restart: unless-stopped
networks:
- caddy
ports:
- 0.0.0.0:80:80/tcp
- 0.0.0.0:443:443/tcp
- 0.0.0.0:443:443/udp
environment:
- TZ=Europe/Berlin
- DESEC_TOKEN=CHANGEME
volumes:
- ./required/Caddyfile:/etc/caddy/Caddyfile
- ./config:/config
- ./data:/data
authelia:
container_name: authelia
image: authelia/authelia:latest
restart: unless-stopped
networks:
- caddy
ports:
- 9091:9091
environment:
- TZ=Europe/Berlin
volumes:
- ./required/configuration.yml:/config/configuration.yml:ro
- ./required/users_database.yml:/config/users_database.yml:ro
- ./required/db.sqlite3:/config/db.sqlite3
### use pre-defined external Docker network: docker network create caddy
### db.sqlite3 needs to exist before first container start, can be created with: touch ./required/db.sqlite3
### Caddy config can be quickly reloaded with: docker exec -w /etc/caddy caddy caddy reload
### changes to Authelia files require its container to be restarted
required/Caddyfile
{
debug
http_port 80
https_port 443
email mail@example.com
# acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
acme_ca https://acme-v02.api.letsencrypt.org/directory
}
*.example.com {
tls {
dns desec {
token {env.DESEC_TOKEN}
}
propagation_timeout -1
}
@authelia host auth.example.com
handle @authelia {
forward_auth authelia:9091 {
uri /api/verify?rd=https://auth.example.com
copy_headers Remote-User Remote-Groups Remote-Name Remote-Email
}
reverse_proxy authelia:9091
}
### example of a basic site entry
@matomo host matomo.example.com
handle @matomo {
forward_auth authelia:9091 {
uri /api/verify?rd=https://auth.example.com
copy_headers Remote-User Remote-Groups Remote-Name Remote-Email
}
reverse_proxy matomo:8080
}
### example of a site entry when target is HTTPS
@proxmox host proxmox.example.com
handle @proxmox {
forward_auth authelia:9091 {
uri /api/verify?rd=https://auth.example.com
copy_headers Remote-User Remote-Groups Remote-Name Remote-Email
}
reverse_proxy 192.168.50.75:8006 {
transport http {
tls_insecure_skip_verify
}
}
}
### Fallback for otherwise unhandled domains
handle {
abort
}
}
Those values depends on the use case. You can set min and max values and fine tune as the need occurs. Some extensive information are discussed in these links:
Thanks, I saw the last link when I first set this up, but not the first two. I'll go through them and see if I can find the sweet spot.
It's hard to tell because while I'm the only user using my Gitea repo website, which is pretty much your own personal Github. However, from what I've read, even though there may only be one or two users, the usage of Elastic greatly depends on how much code it has to cache. Then when you search for something, Elastic has to go through all that code.
So from what I understand, the more code you have in a repo, the more Elastic has to work, which makes figuring out the memory a bit of a random gamble.
I haven't had first hand experience with gitea, but there would be some fine tuning that might ease the memory usage. What backend have you deployed ? You can make some config adjustment to it. If memory constrained, then swapiness could be set and any monitoring could be disabled or kept bare minimum. I read somewhere its useful to pprof https://github.com/google/pprof to get some insight about visually test memory usage, though haven't used it.