BIG Data volumes

    • 57 posts
    December 23, 2022 7:24 AM EST

    How do you handle big data volumes here?

     

    Videos is the toughest since you usually would need

     

    1. Script server

    2. Storage server

    3. Conversion server

     

    I know that SE probably do not support this setup from basic, and SE was never supposed to be a tube script.

     

    Still- what is best way to handle large databases? I will push away the tube side of things to a separate script (it is a must, SE and tube scripts have two different purposes). User databases will be shared via SSE for users.

     

    you who have bigger sites with +100 TB of data, how do you handle it?

     

    CDN is pretty nice overall like ucdn or storage with wasabi (dont like adult content though).

    Me  I am working with larger server for +40 TB for SE script and when that is not enough i most likely will USE UCDN who charges 19 USD per TB of storage and 4 USD TB per CDN streaming, it is pretty fair pricing.

     

    How do you guys who run bigger sites resolve above?

    • 303 posts
    December 30, 2022 6:03 PM EST

    Find a conversion site that could help reduce the video file sizes to reduce storage? 

    I used to use Kraken.io for my images.