Hey all! Firstly a huge thanks in advance to anyone who spends time responding to this.

So I’m working on my MVP which I’m about to launch (in its simplest form this is an AI based news aggregator)

To date my server set up has been:

  1. data storage, scraping, and app API calls to my digital ocean server. This is a 2GB memory, 1 AMD vCPU 50 GB disk server running LAMP on Ubuntu 20.04

  2. All my AI LLM work where I preprocess and clean text, locally run LLMs from hugging face is done through a Scaleway PLAY2-PICO instance.

A few issues I’m facing:

  1. The api calls to the digital ocean server are incredibly slow. Takes 5 seconds to load posts and I’m the only one using the app.

  2. The scaleway server processes for LLMs just get killed I assume due to memory issues or whatever it is.

So now to the question. What is the server architecture / providers you guys use? It needs to be able to deal with large data tables in MYSQL quickly as well as run large LLM models as well (the two don’t need to be the same set up)

Much appreciated!

  • maxip89@alien.topBEnglish
    1·
    1 year ago

    5 seconds?

    For me it looks like that you query to your database each item (Ping pong between application server and database server, the latency between them sums up by the number of items in the database). Some junior developers use ORM layers still they find out they have such a problem. Just search n+1 problem, maybe you can fix it.

    this is not infrastructure related (or you use that much less memory and force your system to swap).

    hope this helps.