Posted on

PHP file upload to a Google storage bucket

Download

bucket_upload_1.0.php.gz

Google setup & use

1- Create a storage bucket for the script to upload into

 

Go to the Google Cloud Console, click on “Storage”, “Browser”.

 

“Create Bucket”

 

Give it a name and click “Create”.

 

2- Create a service account for the script

Expand the “IAM & admin” section, click on “Service accounts”.

 

Click “Create Service Account”.

 

Give it a name, check “Furnish a new private key”, JSON, and click “Save”.

 

Save the JSON credentials file which you are prompted to download into a safe location.

3- Grant “Object Creator” permissions on the bucket to the service account

Go back to the storage bucket you created

 

Edit its permissions

 

The JSON credentials file you just downloaded contains the email for the service account you created, copy it.

 

And paste it into the “Add members” field, select the permission to be “Storage Object Creator”. This service account doesn’t need permissions for anything else than dumping files in there. Not even viewing them.

 

Optional: if you want the files uploaded by the script to be publicly viewable, add the permission “Storage Object Viewer” to the user “allUsers”. Accounts are all referred to by email in Google land, but there exist special keywords such as “allUsers”.

Done with the Google setup 🙂

4- Running the script

If you haven’t already, download the script at the top of this page. Decompress it and edit the config at the top.

$credentials_file_path is the full path to the JSON credentials file you got from Google when you created the service account. It should be a secure location.

$destination_bucket_name is the name of the bucket you created

$access_token_cache_file_path is a location where Google’s OAuth tokens are cached, it too should be a secure location.

Run the script with only 1 argument being the file you want to upload. The script can also be included and used outside of CLI, in that case simply call the upload( $filename ) function.

The script returns the URL to the file in the bucket.

Voilà:

Posted on

A Universal Caching Algorithm for PHP using Memcached

Here is an elegant way to use the same caching logic for all function calls which should have a cache. With the proliferation of 3rd party APIs I was quite happy to find a way to address them all with a single mechanism.

function expensive_third_party_call( $param1, $param2 ) {
    // universal caching algorithm header
    $result = memcached_retrieve( __FUNCTION__ . serialize(func_get_args()) ) ;
    if( $result!==null ) {
        return $result ;
    }

    // this is where the third party call actually happens, if we are hit the cache missed
    $to_return = /* some super complex and time consuming logic, throw in a couple of web calls*/ ;

    // universal caching algorithm footer
    memcached_store( __FUNCTION__ . serialize(func_get_args()), $to_return, CACHE_TIMEOUT ) ;
    return $to_return ;
}



////////// helper functions bellow //////////
$m = false ;

function memcached_retrieve( $key ) {
    global $m ;

    $new_key = md5( $key ) ;
    if( $m===false ) {
        $m = new Memcached() ;
        $m->addServer( 'localhost', 11211 ) ;
    }
    $temp = $m->get( $new_key ) ;
    $result_code = $m->getResultCode() ;
    if( $result_code==Memcached::RES_SUCCESS ) {
        return $temp ;
    } else if( $result_code==Memcached::RES_NOTFOUND ) {
        return null ;
    } else {
        echo "error: can't retrieve memcached key {$key} with result_code {$result_code}" ;
    }

    return null ;
}



function memcached_store( $key, $data, $timeout ) {
    global $m ;

    $new_key = md5( $key ) ;

    if( $m===false ) {
        $m = new Memcached() ;
        $m->addServer( 'localhost', 11211 ) ;
    }
    
    // a little heavy handed but we use null to represent that nothing was found in the cache so we can't have this be the data
    if( $data===null ) {
        $data = false ;
    }

    $m->set( $new_key, $data, $timeout ) ;
    $result_code = $m->getResultCode() ;
    if( $result_code!==Memcached::RES_SUCCESS ) {
        echo "error: can't store memcached key {$key} with result_code {$result_code}" ;
        return false ;
    } else {
        return true ;
    }

    return false ;
}

Requirements

  • apt-get install memcached php-memcached
  • make sure to define CACHE_TIMEOUT

Functioning principle

Using PHP’s awareness of the current function it is in along with the parameters which are passed to it, we derive a unique key which is used so store and retrieve from the cache.