Auto delete comments older than X time to reduce ram usage.

[leem32]

Looking at my server resource usage which has 12GB Ram, I notice Remark42 server is now using 8.8% of total Ram.

I don't remember Remark42 using such a high amount of Ram before (I would definitely have noticed), so I'm assuming this has increased over time based on the number of comments posted on my site. Is this correct? will Remark42 Ram usage increase based on number of comments Remark42 is holding?

Is there any way to decrease the amount of Ram Remark42 is using?
One way could be to automatically delete comments older than X time.
There isn't much value for my site in comments older than 12 months, so I would like Remark42 to delete them.

Its not really practical me deleting them manually because they're spread over hundreds of pages.

Note: Also Remark42 server on my dev server only uses 0.6% Ram of 4GB total Ram, so total comments is obviously causing this huge increase in production server.

[umputun]

pls, use discussions for questions like this.

moved

[umputun]

I have several production instances of remark42 personally, one of them running on the site with tens of thousands of comments and db size is about 300M . It is running on 2g instance and utilizing 220M ram.

I'm not aware of any memory leaks in remark42. Because this is a golang process and it does memory allocations, garbage collection decides what to keep in ram and for how long. The only parameter you can set in go is GOGC (via env variable)

The GOGC variable sets the initial garbage collection target percentage. A collection is triggered when the ratio of freshly allocated data to live data remaining after the previous collection reaches this percentage. The default is GOGC=100. Setting GOGC=off disables the garbage collector entirely.

There are two parts of remark42 utilizing relatively large chunks of memory:

• cache: used to keep prerendered responses. Size of the cache controlled by the user, see --cache.max.* params
• boltdb internal memory: this is not a cache and not a traditional memory allocated, but rather a memory-mapped file.

Nothing else here should allocate noticeable chunks of memory. I don't think there is an actual problem here but rather a GC observing your memory and keeping it because you have it unused.

There isn't much value for my site in comments older than 12 months, so I would like Remark42 to delete them.

I don't see how this will reduce memory footprint

[paskal]

@leem32, could you please share the method of calculation you used for remark42 process RAM usage investigation and the actual numbers you got using it? RAM calculations in Linux are not always intuitive, and I would like to verify that the number we are talking about is actual RAM usage of the process and not cache that could be reused by OS at will.

I propose to add top output to the stats you will provide. To do that, find remark42 PID using ps aux | grep remark (should be in the second column), then run top -p with PID you found, and then press e once to switch to megabytes in the memory usage. You'll get a picture similar to this one (the process is different in my case intentionally), I wonder how it will look like in your case:

[leem32]

@paskal I use Glances for checking RAM which takes the linux cache thing into account.

Glances:
In the containers section you can see Remark42 is using 1.04G. Also in the processes it shows Remark42 server using 8.9% Memory.

PS_mem package also shows same result:

Top also shows same memory usage for Remark:

Free shows the Linux cache memory like the link you provided:

[paskal]

Thanks! That's interesting. When I have spare time, I'll try to prepare a Docker image for you, which will show us precisely where memory is used. Maybe we'll find some ineffectiveness that I could eliminate.

[leem32]

@paskal OK thanks :) I was going to try reloading the container and Docker, but if that fixes it we'll never know what the problem was so I will wait to use the container you create when you're ready.

[paskal]

We won't be able to intercept already running binary for this data without it being re-built with tracing built into it, so I think it's safe to restart it at will. Would you mind letting us know the memory usage after a day after the restart, as opposed to 78 days uptime now?

[leem32]

No problem, so you would just like me to restart Remark42, wait 24 hours and let you now the memory usage at that point, correct?

[paskal]

Correct, that's the idea.

[leem32]

After restarting, the Remark42 memory usage is now very low. I'll post the top again in 24 hours.

[umputun]

@paskal for convenience I have added profiler support to go-pkgz/rest (master only for now)

[leem32]

Good news, 30 hours later and no change.