Community

Forums » SEPHP Help and Tips

Separating audio/video/photoson - S3 and Kraken Optimize

    • 211 posts
    January 28, 2019 1:20 PM EST

    Does anyone have the step-by-step process and correct way

    #1 To store all audio/video/photos in a separate S3 bucket

    - Does this affect performance?

    - Will this work NGINX reverse and CDN correctly?

    #2  How do we optimize the photos with Kraken.io?

    - do we need to change code?

    Kraken says 

    "you would need to relay your image url to us (where your image is stored right now) through a request. The image then gets kraked and pushed to your S3 bucket. "

     

    How do we relay these images?

     

    Thanks!!

     

    • 3537 posts
    January 29, 2019 6:19 AM EST

    Did the thread for Kraken help with this at all? Perhaps ask the question in that thread or point to this thread in that thread so the OP of that one can perhaps help here.

    • 211 posts
    January 30, 2019 5:56 AM EST
    Donna said:

    Did the thread for Kraken help with this at all? Perhaps ask the question in that thread or point to this thread in that thread so the OP of that one can perhaps help here.

     

    Hi Donna,

    I did add this post to the other existing thread

    Had already asked a couple of qtns earlier , but it went to the ether :) lol

     

    Hopefully will get some answers in the upcoming week.

    Tahnks !

    • 3537 posts
    January 30, 2019 12:03 PM EST

    Hope you get an answer. :) 

    • 177 posts
    February 8, 2019 8:34 PM EST

    orry late to the party here. 

     

    I dont know how to store video or audio on aws so cant help out there. 

     

    To store your images to S3 you will need to do it in the admin panel for storage, 

     

    http://community.socialengine.com/forums/topic/1472/image-optimisation-kraken-io

     

    My post in this thread is the steps taken to get this working. Once you have your API from Kraken you just need to replace the details on the script i posted with you own. 9 so yes you will need to modify the code to have your details. 

    The script will be triggered via the lambda function and you would be required to create X amount of trigger for file types ( .gif .png .jpg etc ) Kraken only reduces images, it wont compress audio or video. 

    Here is my lambda trigger setup 

     

    looks daunting at first but its not once you've tried it a few times. 

     

    Lambda code input 

    And again the code to place into lambda

    var kraken = require('kraken') var https = require('https') var querystring = require('querystring') exports.handler = (event, context, callback) => { let records = event.Records for (let i = 0; i < records.length; i++) { getdata(records) } } //This is the lambda function to store images in s3 bucket function getdata (file) { let destination = file.s3.object.key //destination set to object key from s3 bucket let s3_bucket = file.s3.bucket.name // let imageSource = destination.split('/') imageSource = imageSource[imageSource.length - 1] if (imageSource.indexOf('rl-') === 0) { return } if (!(destination.indexOf('.jpg') > 0 || destination.indexOf('.png') > 0)) { return } var Kraken = require('kraken') var kraken = new Kraken({ 'api_key': 'REPLACE ME', // kraken.io api key 'api_secret': 'REPLACE ME' // kraken.io api secret key }) var bucketInfo = { url: 'https://s3.{location}.amazonaws.com/{bucketName}/' + destination, // replace bucketName and location. // example would be https://s3.eu-east-3.amazonaws.com/my-bucket/ wait: true, lossy: false, resize: { // resize image according to your needs 'width': 800, 'height': 600, 'strategy': 'auto' }, 's3_store': { 'key': 'REPLACE ME', // replace bucket-key with your key 'secret': 'REPLACE ME', // replace bucket-secret key with your key 'bucket': 'bucketName', // replace bucketName with your name 'path': destination.replace(imageSource, 'rl-' + imageSource), 'region': 'eu-west-1' // replace bucket-region with your region } } //Callback function from server and failure message kraken.url(bucketInfo, function (data, status) { if (status && status.success) { } else { console.log('Fail. Error message: %s', JSON.stringify(data)) } }) };

     

    I dont have SE installed at the moment but the way to transfer your files to your s3 bucket guide if you do know it this link https://support.socialengine.com/php/customer/portal/articles/1636682-how-to-use-the-cdn-storage-feature

     

     


    This post was edited by PeppaPigKilla at February 8, 2019 8:38 PM EST
    • 177 posts
    February 8, 2019 8:49 PM EST
    playmusician said:

    Does anyone have the step-by-step process and correct way

    #1 To store all audio/video/photos in a separate S3 bucket

    - Does this affect performance?

    - Will this work NGINX reverse and CDN correctly?

    #2  How do we optimize the photos with Kraken.io?

    - do we need to change code?

    Kraken says 

    "you would need to relay your image url to us (where your image is stored right now) through a request. The image then gets kraked and pushed to your S3 bucket. "

     

    How do we relay these images?

     

    Thanks!!

     

    Forgot to answer couple others 

     

    So does this affect performance? Yes in a positive way. 

    Reverse nginx proxy and CDN ? Yes , i used engintron with whm/cpanel with cloudflare cdn, works fine. 

    Kraken says --- This is their manual process. Get your API key and it will work fine through a lambda function. 

    Due to to automatic process of SE uploading your images to S3, every time an image gets uploaded with a trigger extension ( .png .jpg ) then the lambda function will execute the script, the script will use your Kraken API credentials and your S3 bucket details to shrink the file size and store the image. 

    SE replaces the URL of the image with that of the url from your bucket... https://aws.eu-west-2.amazon.com/my-bucket/image.png , you can set this to be masked with your own domain if you choose. 

    ^^ This is how the images would be relayed on your site. 

    Old instructions so the interface will be different but to achieve this http://carltonbale.com/how-to-alias-a-domain-name-or-sub-domain-to-amazon-s3/

     

    I myself didnt really care about the url. 

    • 211 posts
    February 9, 2019 10:26 AM EST

    Thanks PPK :)))

    Its a lot, so gonna dive into it and test it out.

    Thanks for your detailed help.

     

    My first step is to separate /store (audio/video/images) to an s3 bucket 

    I am going to try setting up just the /public folder to s3 to see how that works

    and if it will store the media(audio/video/images)  separately and have a functioning site.

     

    I do have  a kraken api so will test that and lambda with your steps (which are super clear to do)

    Thanks again !!!

    • 177 posts
    February 9, 2019 11:07 AM EST

    Fantastic, 

    I setup a fresh SE installation today and here you can see with the above setup my kraken api is in use 

    And the lambda monitoring shows usage 

     

    its only a dot because its only started today. 

     

    Remember your bucket name is created in the storage settings in Social Engine, not in AWS. SE sets it up with the right permissions. 

    • 211 posts
    February 9, 2019 11:32 AM EST

     awesomenesss :L_)

    cant wait to make it all work, 

    looks great !