we have a Problem with audio recordings in the app. Randomly they just fail to load. Sometimes the last, sometimes all, sometimes a few. Restarting the app usually helps for some time until switching channels. All ~100 users have this problem. On desktop everything works fine.
Images and all other files also always work.
To me it seems that the recordings dont get cached on the phone. As they all seem to download one after another after restarting the app. Which can take some seconds for each (If they even load)
Someone have some idea what is causing this?
Yes only on mobile. But moving from caddy to nginx and then turning of keepalive seems to have fixed the issue as of right now. Maybe the worker_connections limit was just exceeded. I dont know.
Because what seems to happen is the client still downloads all files even after switching channels. So if someone opens up a chat with 50 audio files and quickly moves to another one the 50 still load in the background and all pending requests still get finished up by the server. Wouldnt it be better that a channel switch cancels those?
Now the next problem is that the ram fills up with all those audio files and eventually the App freezes. The browser crashes also. It takes about 50 channel switches so its not extremely problematic but there seems to be some kind of issue with the garbage collection.
Dont know if there is a solution or this is a general problem that needs to be reviewed.
Have tried about everything to fix the problem. I tried it with docker, I tried the different storage options. After loading too many different filese nginx just closes the connection and nothing is loaded until app restart. At least now I have found some nginx configuration that allows the connection to be reset after some timeout while loading the files so after switching channels its actually starts loading files again. At least better than closing the app everytime this happens.
But what I really dont understand is how caching works in the app. I mean the pictures and voice files (aac or m4a) are getting cached by the browser. Nginx is configured correctly for caching those. Does work flawlessly in the browser.
But what does the app do? With every channel switch all the files get loaded again, takes time and I also see nginx sending those completly again. Nothing seems to be really loaded from cache in the app compared to browser.
But what happens when switching channels a lot of times and loading the files again and again is that the app crashes after sometime because seemingly the ram gets completly filled up with alle the files.
With a handful of audio files in the channel thats completely okay. But entering a channel with 100+ audio files leads to transfering 50MB+ every time.
Am I missing something? I want the app behave exactly as the browser. Opening a channel instantly loads all files from cache so nothing gets transfered
Just setup a new instance on a new server and problem persists there as well. For good measure i also tried it with apache. Problem is only in the app. Mobile browser and Desktop browser are all caching the m4a files. So when switching a channel it does not reload all the stuff from the server again.
meanwhile big images (3mb) are definitly getting cached in the app, so no new request is send upon channel switch.
Probably no one ever noticed because with normal use cases like 5 30sec recodings its pretty much instant because its less than 2mb getting loaded in this case.
Another solution would probably also not loading all the audio files upon entering, just loading the one that gets clicked.
hope someone has some time checking this problem. Maybe I have some time for compiling the app so I can do it myself.
I think I know whats up. Its the m4a format that does not get cached. If I spam a channel with normal mp3s they get cached in the app.
But sadly recording in the app is m4a while in the browser its mp3
Edit: luckily that gives us a dirty option to fix it at least in normal File-System storage. Converting the m4v files to mp3 is working. The app then caches all mp3 flawlessly.
So for the time beeing a quick fix like converting the newst files in the folder every couple minutes is enough. like that:
for i in *;
do name=`echo "$i" | cut -d'.' -f1`
if [ $(find /var/snap/rocketchat-server/common/upload_bkp/ -cmin -5 -type f -name "${name}" 2>/dev/null) ]
then
echo "$name"
ffmpeg -v 5 -i "$i" -acodec libmp3lame -ac 2 -y "${name}.mp3"
mv -f "$name" "${file%.mp3}"
fi
done
App still crashes when scrolling up and loading hundreds of files along the way. But unlikely that it happens intentionally. Still maybe there is some memory leak
