Hey all! Firstly a huge thanks in advance to anyone who spends time responding to this.

So I’m working on my MVP which I’m about to launch (in its simplest form this is an AI based news aggregator)

To date my server set up has been:

  1. data storage, scraping, and app API calls to my digital ocean server. This is a 2GB memory, 1 AMD vCPU 50 GB disk server running LAMP on Ubuntu 20.04

  2. All my AI LLM work where I preprocess and clean text, locally run LLMs from hugging face is done through a Scaleway PLAY2-PICO instance.

A few issues I’m facing:

  1. The api calls to the digital ocean server are incredibly slow. Takes 5 seconds to load posts and I’m the only one using the app.

  2. The scaleway server processes for LLMs just get killed I assume due to memory issues or whatever it is.

So now to the question. What is the server architecture / providers you guys use? It needs to be able to deal with large data tables in MYSQL quickly as well as run large LLM models as well (the two don’t need to be the same set up)

Much appreciated!

  • thewanitz@alien.topOPBEnglish
    1·
    11 months ago

    Thanks! Sorry for the noob question but what would you need to know as part of the profiling?

    • xasdfxx@alien.topBEnglish
      1·
      11 months ago

      what would you need to know as part of the profiling

      what’s taking the time

      Obvious things

      1 - your server is out of ram (free -g; ps aux --sort -rss)

      2 - your server is serving a ton of requests (what tho)

      3 - database slow

      4 - database full

      5 - db out of connections (are you using connection pooling?)

      6 - poor choices inside whatever php you’ve written

      7 - internet between scaleway / DO

      That said… all this is optimized to be as cheap as possible. Value your time, use bigger servers, and go figure out if you’ve created any value. If you’ve created value, hosting costs can be optimized.