Community

Forums » SEPHP Help and Tips

Amazon S3 as default storage?

    • 8 posts
    October 11, 2017 2:43 PM EDT

    Hey all,

     

    I spent the better half of 9 days transferring 200+GB worth of files to an Amazon S3 bucket via the transfer tool under "Manage Storage Services". All of the files transferred fine like expected, but now when selecting S3 as my default storage, files take an extremely long time to upload. When I revert back to my server as the storage, everything uploads quickly.

     

    Using S3 as my default storage, a 1MB file will take a minute or so to upload, a 3MB maybe takes 3-5minutes, a 20MB file takes 5+ minutes to upload and so on. I love the idea of having this external storage for my site because my members upload a lot of content, but this slow upload using S3 is not a viable option. 

     

    So, is anybody else using S3 as their default storage? How is it working for you when uploading files? Slow? Fast? I'd love to get some input on this.

    • 681 posts
    October 12, 2017 4:23 AM EDT

    I think it depends on the region you selected. I recall seeing something about this for someone else. https://forums.aws.amazon.com/thread.jspa?threadID=116513 try a traceroute and there's one mentioned by Amazon in that thread. See if it shows any issues. We use Amazon here and at our demo and hadn't noticed an issue.  

    • 8 posts
    October 12, 2017 9:09 AM EDT

    Hi Donna,

     

    Thanks for the response.

     

    I am located in Dallas, my server is located in Chicago and my bucket is located in N.California. Maybe moving my files over to a bucket closer to my server (Ohio?) would improve a tiny bit? I'll review that thread in the link you provided.

     

    It should also be noted that I'm using SE 4.8.12 on PHP 5.6 (apache 3). Not sure if these outdated versions are adding to the problem but I won't be positive until Saturday when I upgrade to PHP 7.x (apache 4) and the latest version of social engine/plugins. I'll follow up to this thread once everything is upgraded.

     

    But in the mean time, you said the social engine demo uses S3 as the default storage location? Because browsing around the demo and viewing source on profile pictures, uploaded music etc. shows the files to be uploaded locally. 

     

    Example:

    Profile photo: https://demo.socialengine.com/profile/gary  --> https://demo.socialengine.com/public/user/1a/01/f79e96a5e722ebe1c115e678c8fc1528.jpg

    Music: https://demo.socialengine.com/music/2/travelin-039-man  --> https://demo.socialengine.com/public/music_song/35/01/b59af033c23a2ba8bb3d330eda39677e.mp3

    • 681 posts
    October 12, 2017 9:20 AM EDT

    You should upgrade as there were a lot of improvements since that version. I'll double check the demo as I thought it was using S3 like the rest of our sites. 

    • 681 posts
    October 12, 2017 9:23 AM EDT

    Ok yeah it uses Amazon but since it gets reset, the main files in there for the demo content are stored elsewhere due to the reset each day. Newly uploaded stuff for people testing should be on Amazon if you test by signing up and loading a photo or something. This site also uses Amazon, just double checked.

    • 8 posts
    October 14, 2017 8:08 PM EDT

    Thanks for the feedback, Donna. I've upgraded to the latest version of SE4 as well as PHP. I will update you all how S3 upload speeds are in a few days, now that I've upgraded.