At iPragmatech, we generally try to avoid building from scratch and tried to reuse the open source component. Our phpfox developers found a wonderful jquery’s plugin jquery-timelinr developed by csslab. This jquery plugin is light, extensible and easy to use. We can easily add our custom style in timeline and just need to send data in it’s desired format. We shall show you how we used this plugin to customize phpfox for timeline milestones.

phpfox milestones plugin

1. Include the jQuery library and this plugin:

<script src="js/jquery-1.6.1.min.js" type="text/javascript"></script>
<script src="js/jquery.timelinr-0.9.js" type="text/javascript"></script>

2. Initialize-it with the default parameters:

<script type="text/javascript">

$(function(){

$().timelinr();

});

</script>

3.  HTML markup must be as follows:

<div id="timeline">

<ul id="dates">

<li><a href="#">date1</a></li>

<li><a href="#">date2</a></li>

</ul>

<ul id="issues">

<li id="date1">

<p>Lorem ipsum.</p>

</li>

<li id="date2">

<p>Lorem ipsum.</p>

</li>

</ul>

<a href="#" id="next">+</a> <!-- optional -->

<a href="#" id="prev">-</a> <!-- optional -->

</div>

You can also configure lots of setting in timeline view with the initialization of timeline. Feel free to send you feedback for this article. We offer end-to-end phpfox expertise, including consultancy, building new modules, customize existing module features, migration from older versions and performance tuning.

So problem was if a user is visiting some page which user can view only and decides to login in order to do any action on this page. So whenever user clicks on SocialDNA facebook connect button from the socialengine community site, It would open a dialog box in which user can enter facebok credentials. After authenticating the facebook credentials, it redirect user to the home page not the page where user came. This can be very annoying experience for the community user so we decided to change this behaviour. Our socialengine developers dig out the source code of SocialDNA plugin and found a way to customize the plugin. Our developers found that its pretty easy to change the behavior as only need to get the previous page referrer and that can be retrieved from Server’s Parameter. So we made changes in controlls AuthController.php and QuickSignupController.php  as below:
Change

return $this->_helper->_redirector->gotoRoute(array('action' => 'home'), 'user_general', true);

to

return $this->_helper->redirector->gotoUrl( $_SERVER['HTTP_REFERER']);

With the above changes, plugin should able to work as desired behavior i.e. user should be redirected to the previous page on the socialengine.
We offer end-to-end SocialEngine expertise, including consultancy, implementation, customization, migration from older versions of SocialEngine and performance tuning. Feel free to contact us if you have any query related to socialengine development.

Resources:

• SocialEngine PHP Community Software – Create your own community website
• Social DNA

The purpose of creating phone plugin was to connect the community and non-community users with each other and keep them updated about each other. The community owners can monetize their community by selling this feature to the community users. So it would be win-win situation for both the users and owners. This plugin allows the user to add non-community users in their contacts list and allow to send them SMS. The plugin also keep track of the SMS sent to the users so user shall be able to track the sent SMS.Currently SMS plugin uses clickatell and twilio SMS gateway to send out the SMS efficiently.

Currently this SMS plugin is in still development phase as our client is looking for funding to complete this plugin. This plugin can be extended to support following features or any customize feature:

• Allow community users to send SMS to community friends
• Send status updates to community friends
• Receive SMS from the contacts
• Integration with other popular SMS gateway like tropo, voxeo etc.
• Admin control and reporting
• Integration with payment gateway
• Integration with credits plugin
• Send notifications for events to the users
• Send promotion coupons from the advertisers

Feel free to contact us if you think this plugin would help your community to grow and we can discuss about the value proposition for this plugin.This plugin have also capability to record the voice message and send it to user’s mobile as voice mail.

Demo:

Please check the status of online plugin here

user:sms@ipragmatech.com

pwd:123456

 Photo Gallery:

Resources:

• SocialEngine PHP Community Software – Create your own community website
• How to Monetize Social Media

Recently Hire-Experts comes up with the virtual gifts plugin for the socialengine and its amazing plugin to send photo,audio and video gifts. The empower the community users to connect well with their friends.

“You can create as many Virtual Gifts as you would like and link them to any holiday. It can be not only a photo, but also a video or even a music. It automatically suggests members to send a gift on their friends birthday or on holidays – it is simple and fun.”

We also tried this plugin on one of our client’s community and it works fine with basic functionality. As soon we tried to test every feature of the plugin, we encountered issue while uploading a virtual photo gift. We were not able to upload the photo and progress bar for upload remains standstill on 0% event after few minutes.

We tried to upload the audio or video gifts to make sure that the issue is with only with photo gift but these features were also not working.  We tried features based on fancyupload in socialengine and concluded that there is definitely some issue with the plugin as other features on socialnegine works fine. So we decided to fix the issue our self as our client insisted us to launch this plugin as soon as possible.

We looked at the code and found that the template file from HE plugin doesn’t seem to set the cross-domain policy in the javascript function.  We removed the MAX-CDN reference from the socialengine and tried to test the upload.  The functionality worked this time and this verified that the plugin have cross-domain policy issues. We add following lines in the file ‘_FancyUploadPhoto.tpl’ while initializing FancyUpload for the upload:

 // set cross-domain policy file
 policyFile : '<?php echo (_ENGINE_SSL ? 'https://' : 'http://')
        . $_SERVER['HTTP_HOST'] . $this->url(array(
          'controller' => 'cross-domain'),
          'default', true) ?>',

After the above changes, we were able to upload photo gifts. We applied the same changes in the audio and video templates and they worked as well.

We hope HE guys would incorporate this change in their plugin and come up with a patch to support uploading for cross domain. Feel free to give us your feedback and contact us if you wanted to build your own social community based on socialengine.

References:

• SocialEngine PHP Community Software – Create your own community website
• Hire-Experts

Voice Message

• Monetize your community
• Send and receive voice message
• Easily customizable with any telephony vendor
• Extensible support for SMS
• No requirement for streaming server

We have been working on a module which allows socialengine community users to send recorded voice message on user mobile phone. In some countries there is limited access to internet due to their government policy though most of the people in those countries have access to mobile phone. So its easier for a community user to send voice message or song to their friends and relatives without requiring the access to internet.

We knew that it would be challenging job and the first task was to find out a solution to record the voice from a website. There were multiple solutions like implementing the client for audio streaming server like RED5 or Flash Media Server. Our client requirements was to record the voice message for 2-3 mins and adding audio streaming infrastructure was overkill. So we decided to use a very simple solution which allow us to record the voice and able to send it to our server. Luckily we found a wonderful open source plugin for voice recording by sajith. This plugin allow us to record the voice and also allow to review the recording.  This plugin also allow us to send the voice to the server in wave format. You can found many utilities for linux like ffmpeg or  lame which can easily convert the wav files to mp3.

There were some missing features & bugs in the plugin. The main problem with the plugin was that it can’t work with any third party javascript’s framework other than jQuery. Since socialengine uses prototype as its javascript framework, the recording plugin didn’t worked at all.  The other major issue with the plugin was that you cannot add parameters on the fly to the server. The easy fix could be to make two calls, one for the file and other for other parameters. Also there was issue with this approach as callback for upload completion was not working properly. So we decided to fix those issues and contributed those fixes back to the project for the benefit of the socialengine community.

Now the next challenge was to send the voice message to the user’s actual device. There are many vendors which can provide telephony infrastructure to send the voice message to user’s device. So one can choose the appropriate vendor for the specific country. Our client decided to use casiencuba to  send the voice message to the user’s device.

Now the last part of the challenge was to receive the voice mail from the user and publish it to socialengine. For this purpose, we exposed a service to vendor who can publish the voice message to the socialengine community. Once socialengine service gets the reply message from the vendor, voice mail plugin stores the information in the system. Community user is notified about the voice message and user shall be able to play the voice message.

This module is extensible enough to support the SMS feature as well. Feel free to contact us if you have any question or interested in customizing this module for your need.

As the launch getting closer, we faced issues while configuring the SMTP server settings on phpfox. We entered the relevant information for mail server in the php mail settings but were facing issues like ‘AUTH not accepted from the server’.

We tried to find out the issue with phpfox documentation but could not found the solution though many people have complained about this solution.

So we decided to dig more into the source code and fix this issue. We were able to fix the issue by changing the source code of ‘class.phpmailer.php’. We just replaced the source

public $SMTPSecure    = '';

with

public $SMTPSecure    = 'tls';

After saving this change, we ere able to configure Amazon SES with phpfox. We think the same changes would also work with google smtp server.

We wish you have found this post useful and would help your community to use the power of SES for phpfox. Feel free to send us your suggestions or questions.

In this article, we shall discuss about the opcode and caching solution for the phpfox v3 platform and performance of these solutions on constant user load. We have used apache’s benchmark tool for this purpose and results were eye opener.

performance comparision xcahce apc memcache on phpfox v3

For this Benchmark, we tested 1000 request on the home page with 10 concurrent users. We have run the apache’s tool at least 2-3 times and pick the best case for the particular scenario. We shall not cover the installation of these caching solutions in the article, If someone is interested then please take a look at our previous blog. You can configure caching in phpfox3 as file cache and memcached as discussed in this article. Xcache and APC would be opcode caching solutions for phpfox v3.

For the testing purpose ,we kept the settings for apache webserver and mysql as default so result may be different in case these resources are optimized.Also there was minimal data on the website so result may also vary depending on the contents of the website.

Testing Environment:

• Operating System: 2.6.35.14-106.53.amzn1.x86_64
• Web Server:  Apache/2.2.21 (Unix)
• PHP: PHP 5.3.10
• CPU: Intel(R) Xeon(R) CPU E5430  @ 2.66GHz
• Main Memory: 7652592k

Test Results:

Memcached

Concurrency Level:      10
Time taken for tests:   103.594 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      29972124 bytes
HTML transferred:       29559780 bytes
Requests per second:    9.65 [#/sec] (mean)
Time per request:       1035.940 [ms] (mean)
Time per request:       103.594 [ms] (mean, across all concurrent requests)
Transfer rate:          282.54 [Kbytes/sec] received

APC

Concurrency Level:      10
Time taken for tests:   82.830 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      30110000 bytes
HTML transferred:       29696000 bytes
Requests per second:    12.07 [#/sec] (mean)
Time per request:       828.295 [ms] (mean)
Time per request:       82.830 [ms] (mean, across all concurrent requests)
Transfer rate:          355.00 [Kbytes/sec] received

Xcache

Concurrency Level:      10
Time taken for tests:   86.303 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      30110000 bytes
HTML transferred:       29696000 bytes
Requests per second:    11.59 [#/sec] (mean)
Time per request:       863.026 [ms] (mean)
Time per request:       86.303 [ms] (mean, across all concurrent requests)
Transfer rate:          340.71 [Kbytes/sec] received

Plain php

Concurrency Level:      10
Time taken for tests:   157.491 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      30110000 bytes
HTML transferred:       29696000 bytes
Requests per second:    6.35 [#/sec] (mean)
Time per request:       1574.913 [ms] (mean)
Time per request:       157.491 [ms] (mean, across all concurrent requests)
Transfer rate:          186.70 [Kbytes/sec] received

Conclusion

As per our test cases, Both APC and Xcache provides a great performance boost to phpfox v3 than memcached/Simple File Cache (No Opcode) on a single machine. APC remain faster ~5% better than Xcache, ~25% better than memcached and ~90% better than plan php. So we think using either APC or Xcache shall boost the performance of your social community based on phpfox v3. If you have phpfox v3 running multiple servers then you should consider memcached as caching solution.

• Dynamically Captured Image
• Better Solution for news letter
• Used ImgKit Gem for rails

We found an article on google , which have detailed out the list of supported css/style in different email clients. There are some css support information which seems incorrect but overall conclusion is that it is very time consuming exercise to get proper/desired design of email content or sometime you might not get the desired results at all e.g. ‘radius’ or ‘gradients’ parameters are supported only in html5/css3/webkit.

As email is very crucial part of advertisement campaign as advertiser need to send their ad or deals through email newsletter. This requirement require some out of the box thinking skills so we started to find out some alternative of this css/style based ad. We heavily use webkit css/styles for advertisement components and those displayed perfectly fine on browsers like Firefox,Chrome or Safari but might not work on email clients. After lots of discussion among the team, we got an idea that to take the screenshots on the runtime and send this image in email content easily.

As we were working on ruby on rails, we got a gem IMGKit, which creates JPGs using plain old HTML+CSS. Imgkit uses wkhtmltoimage on the backend which renders HTML using Webkit. ImgKit heavily based on PDFKit . IMGKit is a great tool to convert HTML file/string into image and also supports css file/inline. You need to install wkhtmltoimage with IMGKit so we shall guide you to install the prerequisites and use the gem to generate image by rendering the html on the fly.

Install:

1. IMGKit:

gem install imgkit

2. wkhtmltoimage:

Automatic:

sudo imgkit --install-wkhtmltoimage

install latest version into /usr/local/bin

Manually : If you face any problem with install wkhtmltoimage by command then you can take latest binary from
http://code.google.com/p/wkhtmltopdf/downloads/list and copy at location /usr/local/bin

How To Use:

# IMGKit.new takes the HTML and any options for wkhtmltoimage
# run `wkhtmltoimage --extended-help` for a full list of options
kit = IMGKit.new(html, :quality => 50)
kit.stylesheets << '/path/to/css/file'

# Get the image BLOB
img = kit.to_img
# New in 1.3!
img = kit.to_img(:jpg)      #default
img = kit.to_img(:jpeg)
img = kit.to_img(:png)
img = kit.to_img(:tif)
img = kit.to_img(:tiff)

# Save the image to a file
file = kit.to_file('/path/to/save/file.jpg')
file = kit.to_file('/path/to/save/file.png')

# IMGKit.new can optionally accept a URL or a File.
# Stylesheets can not be added when source is provided as a URL of File.
kit = IMGKit.new('http://google.com')
kit = IMGKit.new(File.new('/path/to/html'))

# Add any kind of option through meta tags
IMGKit.new('<html><head><meta name="imgkit-quality" content="75"...

# Format shortcuts - New in 1.3!
IMGKit.new("hello").to_jpg
IMGKit.new("hello").to_jpeg
IMGKit.new("hello").to_png
IMGKit.new("hello").to_tif
IMGKit.new("hello").to_tiff

Rails Controller Actions

@kit = IMGKit.new(render_to_string) # we can use also Html string

format.jpg do
send_data(@kit.to_jpg, :type => "image/jpeg", :disposition => 'inline')
end

After doing all this set up, IMGKIT was working fine and generating great image from html.But for some cases it was showing an error “QPixmap: Cannot create a QPixmap when no GUI is being used”. Then we tried to find the reason and got that it was happening only for those html which have translate-gradiant(css property). This issue is also listed at here and several people are facing this issue. Then we got that there is an issue with binary of wkhtmltoimage(V 0.11.rc1), we recompiled the latest source code and this time it was working fine for all cases. You can also try wkhtmltoimage(V 0.10.0_rc2) ,this binary also have this fix and works fine.

We would like to have your feedback and would like to answers your query.

• Authentication issue with xcache using fastcgi on apache2
• Simple hack to resolve the issue

We configured the apache server with fastcgi module and powered it with the alternative FastCGI processor manager PHP-FPM.Socialengine community performance improved and was able to take up 200 concurrent users without any glitches or without compromising the loading time. We decided to replace the APC with Xcache(1.3.2-2) as later caching performs better than the earlier caching solution. Please feel free to take a look at our previous blog which helps you to install APC, Xcache and MemCache for your socialengine.

Social Community had no issues once it was configured with Xcache but there were issues while editing the language phrases in the Admin control panel.The page kept on prompting for the authentication for Xcache Administration even if we entered the correct credentials. We even tried to set the authentication flag(xcache.admin.auth=off) but it kept on prompting for the authentication. We found the temporarily work around to resolve this issue by putting socialengine in development mode while making the changes for phrases. Well the issue was that the community have many online users all the time and putting socialengine on development mode would have affected the site’s performance.

So we decided to dig further and found that the xcache, for some reason, doesn’t validate the credentials for fastcgi module. We found a little post on google in which some user mentioned the hack to get rid of the Xcache authentication. So we wanted to share that with you guys as well.

You need to download latest xcache from the site (Assuming you already have installed Xcache)

yum remove php53u-xcache
wget http://xcache.lighttpd.net/pub/Releases/1.3.2/xcache-1.3.2.tar.gz
tar xzvf xcache-1.3.2.tar.gz
cd xcache-1.3.2
vi xcache.c

and change the code from

/* auth disabled, nothing to do.. */
        if (!XG(auth_enabled)) {
                return 1;
        }

to

/* auth disabled, nothing to do.. */
        if (1 || !XG(auth_enabled)) {
                return 1;
        }

Once you are done with the changes, then you need to recompile and install it again

phpize
./configure --enable-xcache
make
make install
cd /etc/php.d/
mv xcache.ini.rpmsave xcache.ini

Now you need to add basic authentication in the apache configuration file to make Xcache administration more secure. Please google and found a way to add basic authentication.
Feel free to send us your questions or issues related to socialengine, we would be happy to answer them.

1. Default Sign up module: Socialengine has five default steps for signup and you can find them in engine4_user_signup table in DB . Here is the default table data:

The steps with the sequence displayed to new user

2. Add or Remove steps from Db: We can disable any default step by changing value in ‘enable’ column. We have added two new steps after first step as:

Custom steps with the sequence displayed to new user

3. Create Plugins for new steps : Now we jump into the source code and do the real work.We just need to create two new plugins for two new steps, we have created two new plugins:
a. User_Plugin_Signup_KidspotFields
b. User_Plugin_Signup_MumsSaysFields

In user -> Plugin -> Signup. You can take the code from default plugin User_Plugin_Signup_Fields and just need to change upper declaration of form and page.

We used:
a. User_Plugin_Signup_KidspotFields:

protected $_formClass = 'User_Form_Signup_KidspotFields';

protected $_script = array('signup/form/kidspotfields.tpl', 'user');

b. User_Plugin_Signup_MumsSaysFields:

protected $_formClass = 'User_Form_Signup_MumsSayFields';

protected $_script = array('signup/form/mumssayfields.tpl', 'user');



Rest of the part of code would be same as User_Plugin_Signup_Fields.

4. Create new forms and Pages: we need to create new forms and view pages for it.
Forms: ‘User_Form_Signup_KidspotFields’ and ‘User_Form_Signup_MumsSayFields’

View Pages: ‘signup/form/kidspotfields.tpl’ and ‘signup/form/mumssayfields.tpl’

You can implement view(.tpl) file according to your need or just add –

form->render($this)

to show only form part.

5. Final step: Now, we want to implement form for new steps.We need to create our own generate function instead of default generate function.

Here we wanted to show to total 80 profile fields into two pages. First page(‘User_Form_Signup_KidspotFields) would have 48 fields and second page(‘User_Form_Signup_MumsSayFields) would have 48-80 fields.

Here is the generate function for ‘User_Form_Signup_KidspotFields:

public function generate()
{
$struct = $this->getFieldStructure();

$orderIndex = 0;

foreach( $struct as $fskey => $map )
{

$field = $map->getChild();

// Skip fields hidden on signup
if( isset($field->show) && !$field->show && $this->_isCreation ) {
continue;
}

// Add field and load options if necessary
$params = $field->getElementParams($this->getItem());

//$key = 'field_' . $field->field_id;
$key = $map->getKey();

// If value set in processed values, set in element
if( !empty($this->_processedValues[$field->field_id]) )
{
$params['options']['value'] = $this->_processedValues[$field->field_id];
}

if( !@is_array($params['options']['attribs']) ) {
$params['options']['attribs'] = array();
}

// Heading
if( $params['type'] == 'Heading' )
{
$params['options']['value'] = Zend_Registry::get('Zend_Translate')->_($params['options']['label']);
unset($params['options']['label']);
}

// Order
// @todo this might cause problems, however it will prevent multiple orders causing elements to not show up
$params['options']['order'] = $orderIndex++;

$inflectedType = Engine_Api::_()->fields()->inflectFieldType($params['type']);
unset($params['options']['alias']);
unset($params['options']['publish']);

//********************************************************
// here is the line where we would add a check for numbers of fields
if($orderIndex < 48) {

$this->addElement($inflectedType, $key, $params['options']);

$element = $this->getElement($key);

if( method_exists($element, 'setFieldMeta') ) {
$element->setFieldMeta($field);
}

// Set attributes for hiding/showing fields using javscript
$classes = 'field_container field_'.$map->child_id.' option_'.$map->option_id.' parent_'.$map->field_id;
$element->setAttrib('class', $classes);

//
if( $field->canHaveDependents() ) {
$element->setAttrib('onchange', 'changeFields(this)');
}

// Set custom error message
if($field->error){
$element->addErrorMessage($field->error);
}

if( $field->isHeading() )
{
$element->removeDecorator('Label')
->removeDecorator('HtmlTag')
->getDecorator('HtmlTag2')->setOption('class', 'form-wrapper-heading');
}
}
}

$this->addElement('Button', 'submit', array(
'label' => 'Continue',
'type' => 'submit',
'order' => 10000,
));
}

You can check the highlighted line of function if($orderIndex < 48), here we added to show first 48 fields.
You need to add this function in ‘User_Form_Signup_MumsSayFields also, but we need to change a line to show fields from 48 to 80. Here is code:

if($orderIndex >= 48 && ($orderIndex < 80)

So this is the procedure we followed and were able to show signup wizard with minimum changes in the socialengine platform. Here are the screenshots after the customizing the signup steps.

Features

• Displays events as a list or within a calendar grid
• Calendar grids can have the ability to change the month displayed
• Accessibility to for events

Displays events as a list or within a calendar grid

Open Browser from App

The simplest case is when you just want to open the browser from your application. You can do this by the function

// Create the URL string
NSString* urlString = [NSString stringWithFormat:@"http://maps.google.com/
maps?q=%@", @"india"];

// Launch the browser!
[[UIApplication sharedApplication] openURL:[NSURL URLWithString:urlText]];

This is a very simple approach, however with several disadvantages. One is that your application is no longer running in the foreground. Also your application does not have any control over the web application that has been opened.

Open link within app

The next way is opening a link within the application. It is easily achieved by using a UIWebView as follows:

// Create the web view
UIWebView* webView = [[UIWebView alloc] initWithURL:@"http://
maps.google.com/maps?q=%@", @"india"];

This can be added to any view in your application. Now we can close the webview from our application when we need to. In addition, we get callbacks for page loading, link clicks and so on.

What if the integration is more complex? Imagine this scenario – the user clicks a button in the native application and you would like to call some JavaScript function in your web application. This is done by the following code:

[webView stringByEvaluatingJavaScriptFromString:
@"alert('javascript called')"];

Not only you can call any JavaScript functions available in the web page being displayed, but you can call any use any JavaScript string, or even submit forms. Now what if we need to get the result of some JavaScript called on the web page into our native application? If an alert is called in the JavaScript on the web page, and listening in the web view as follows.

// In JavaScript
alert('send event')

// In native application

- (void)webView:(UIWebView *)webView
runJavaScriptAlertPanelWithMessage:(NSString *)message
initiatedByFrame:(id)frame {
NSLog(message);
}

The above code would print Send Event.

You can use the same method to do a more generic version of interaction, if you need to do a call from the web view based on what you received from JavaScript.

// In JavaScript
alert('callback();');

// In native application

- (void)webView:(UIWebView *)webView
runJavaScriptAlertPanelWithMessage:(NSString *)message
initiatedByFrame:(id)frame {
[webView stringByEvaluatingJavaScriptFromString: message];
}

Since the message contains callback(), this function will be called on the JavaScript by the web view.

Using JavaScript API

Lastly we come to our interesting use case when there is only JavaScript APIs available from a third party. All you need to do is, create an HTML page which contains the functions and include it with your application. Open this HTML in web view as follows:

NSString * nsstrPath = [ [ NSBundle mainBundle ] pathForResource :
@"MapsApi" ofType : @"html" ] ;
NSURL * nsURL = [ NSURL fileURLWithPath : nsstrPath ] ;
NSURLRequest * nsURLRequest = [ NSURLRequest requestWithURL : nsURL ] ;
[ webView loadRequest : nsURLRequest ] ;

We hope this article shall help you creating mobile app using iOS and HTML5. We would love to hear your feedback so please feel free to add your comments. There are many frameworks which can help you creating mobile app. We prefer Sencha touch though.

We were using the apache + mod php for the socialEngine for one of our clients. Initially when there were less user load then website used to perform well but as user load increases with the popularity of the website, site become slower and sometimes it used to crashed without giving any warning. It was very difficult to found the exact issue for the crash. Initially we suspected that the mysql (MariaDB) can be the main culprit but after installing munin, it become clear that the issue is with Apache. We investigated further and found that apache’s preform MPM uses multiple child processes with one thread each and each process handles one connection at a time. So this resulted in more memory usage for apache as more and more users visited the website.This resulted in system crash and we used to reboot the system manually.  We tried to fine tune the prefork parameters but site was still falling apart.

We had no clue what should we do to stop the site crash. Users started complaining about page not opening or taking too much time to open the page. We thought about moving the static contents (css/js/images) to different server or even though of using Amazon s3. After looking at the articles on google, we tried NGINX as reverse proxy for our Apache’s server. In this mode, NGINX looks like just another Web server to users. Behind the scenes, though the reverse proxy is used to load balance Web requests among several back-end servers, or to provide caching for a slower back-end server. This result in very fast and stable Web Site with same hardware.

There were spikes in the apache's access during the starting of September month when only Apache was used.

Using NGINX as reverse proxy took the most of the load from apache server which resulted in fast and stable website. Earlier accesses/min used to reach ~5K and resulted in more apache’s processes thus more memory usage. Site is stable now as NGNIX handles static contents (js/css/images) while apache needs to process only php scripts.

Apache used to use 150 processes in month of September and resulted into site crash

Memory usages reduced drastically after ngnix acting as reverse proxy

NGINX can deliver 10 times the performance on the same hardware. NGINX is very fast and uses far less resources than Apache. It does this by being event-based. That means it doesn’t spawn new processes or threads for each Web page request. So even as the load increases, memory use remains predictable

The performance can be further increased by using php-fpm or fast-cgi in apache instead of using mod-php or use ngnix+fast-cgi. We tried to follow these approaches but found some limitations with each approach with our requirements. We would love to know more approaches to improve performance of website so please don’t hesitate to comment.

References:

• Apache and IIS’ Web server rival NGINX is growing fast
• Apache vs nginx web server performance deathmatch
• Using Nginx as reverse proxy

Handle HTTP/1.1 401 Unauthorized and SERVICE_NOT_AVAILABLE Error:

• Your “Role account email *” should be your google email account
• Make sure your “Role account email *” was properly filed in the signup form
• Setup your “Role account email *” in the emulator account once. This is not documented in the c2dm documentation but it worked like magic to get rid the error  HTTP/1.1 401 Unauthorized
• In case you have setup you domain with google app (we enjoy google email and other services for our domain) then use option “GOOGLE_OR_HOSTED” OR “HOSTED” while generating Auth code.

We hope using this checklist, you should be able to get rid of the error for android push notification (c2dm). Please feel free to send us your thoughts or questions.

In the previous article, we discussed about optimize performance of socialengine using caching system. In this article, we would discuss about optimize performance of any website by tunning server settings and following best practices for static contents. Using these tips can improve the speed of a website by 30%-70%.  Though we did this improvements for socialengine but these tips can be used for any custom built website built on on any technologies like JEE, Microsoft .NET, PHP or Ruby on Rails. We are sharing these tips so that anyone can optimize performance of their own website. We are more focusing on the checklist approach. You can get the details in references section at the end of this article.

Note: We had followed this approach while optimizing a socialengine 4 website for one of our client.

Enable HTTP compression

Compressing resources with gzip or deflate can reduce the number of bytes sent over the network. Most of the modern browser supports data compression for html, CSS and JavaScript. Downloading of compressed data shall reduce the downloading time and allows your website to open faster. You can easily enable the compression on apache web server by editing config file or changing .htaccess at the root of the application.

Using deflate module

<ifmodule mod_deflate.c>
AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml text/css application/x-javascript application/javascript
</ifmodule>

or using gzip module

<IfModule mod_gzip.c>
    mod_gzip_on       Yes
    mod_gzip_dechunk  Yes
    mod_gzip_item_include file      \.(html?|txt|css|js|php|pl)$
    mod_gzip_item_include handler   ^cgi-script$
    mod_gzip_item_include mime      ^text/.*
    mod_gzip_item_include mime      ^application/x-javascript.*
    mod_gzip_item_exclude mime      ^image/.*
    mod_gzip_item_exclude rspheader ^Content-Encoding:.*gzip.*
</IfModule>

Add expires headers

Adding expires headers do not affect the site load time for a first time visitor but you will be surprised how much the page load time decreases (faster page load) for subsequent page views/return visits from that visitor. Expires header specifies a time far enough in the future so that browsers won’t try to re-fetch images, CSS, javascript etc files that haven’t changed (this reduces the number of HTTP requests) and hence the performance improvement on subsequent page views.
To add expires header to the image, CSS, javascript files add the following to your .htaccess file

#Expire Header
<FilesMatch "\.(ico|jpg|jpeg|png|gif|js|css|swf)">
ExpiresDefault "access plus 2 hours"
</FilesMatch>
or
# Expire images header
ExpiresActive On
ExpiresDefault A0
ExpiresByType image/gif A2592000
ExpiresByType image/png A2592000
ExpiresByType image/jpg A2592000
ExpiresByType image/jpeg A2592000
ExpiresByType image/ico A2592000
ExpiresByType text/css A2592000
ExpiresByType text/javascript A2592000

Minify and combine javascripts

Removing extra spaces, line break, indentation, comments from the files can reduce size of the file and enabling faster downloading. There are tools like JSMin or the YUI Compressor which can be used to minify the javascripts.  Combining multiple javascripts in one file shall allow better compression and lesser http threads from the browser thus other resource can be downloaded in parallel.

Minify and combine css

Minifying CSS has the same benefits as those for minifying JS: reducing network latency, enhancing compression, and faster browser loading and execution.Several tools are freely available to minify JavaScript, including the YUI Compressor and cssmin.js

Use Google’a AJAX Libraries API or Yahoo’s ComboHandler

Yahoo’s Combo Handler and Google’s AJAX Libraries API both serve content from their CDNs and potentially increase the chance that your visitors will already have a file in their browser cache. Neither service serves custom content that you provide. You may wish to use these services to serve popular libraries and Minify to serve your code.

Optimize image size

Use the resized image that fix the content of the page rather than resizing it in the image tag e.g. if you wanted to display 60×60 image while original image size is 480×480 then some developer tries to make change in the image tag by defining the dimensions.Using smaller image at this place would allow faster loading of the image as smaller image size would be much lesser than the original image size. Optimize the image by reducing the unnecessary bytes from the image. There are many  tools like smushit or GIMP which can you used to optimize the images.

Use css image sprite

CSS sprites are a way to reduce the number of HTTP requests made for image resources referenced by your site. With this style, images are combined into one larger image at defined X and Y coordinates. There are many tools like SpriteMe and Sprite Creator which can help you to create sprite of the images and generate css for them.

Serve static content from a cookieless domain

Static content, such as images, JS and CSS files, don’t need to be accompanied by cookies, as there is no user interaction with these resources. You can decrease request latency by serving static resources from a domain that doesn’t serve cookies. This technique is especially useful for pages referencing large volumes of rarely cached static content, such as frequently changing image thumbnails, or infrequently accessed image archives. We recommend this technique for any page that serves more than 5 static resources. You can use put your contents in separate server(may be subdomain) or file storage service like Amazon S3.

Use alternative web server

There are plenty more good alternatives for the Apache web server, so if you have a more specialized need, take a look at them, maybe you’ll find something that suits you better. Many of these alternative web servers are free to use. Nearly all offer a lightweight, highly scalable server solution. Some even outperform Apache. The 5 web servers we will be looking at as real alternatives to Apache are:

• Nginx
• LiteSpeed
• Lighttpd
• Hiawatha
• Cherokee

Please feel free to send your suggestions or any other technique which can improve website speed.

References:

• Web Performance Best Practices
• Enabling gzip, deflate compression on your website on apache via .htaccess
• 5 Lightweight Alternatives to Apache and IIS Web Servers

Welcome to the world of  ’Cloud to Device Messaging’ or C2DM supported by google services. This solution provides a  solution to send real time notification to the user on user’s handset. Since Google already uses its service for Calendar or gmail etc. on the android platform so there won’t be extra network usage or battery usage for c2dm.

There are plenty of excellent articles which explain working on C2DM push notification and provide sample code to Run. We feel some points were missing or not obvious in those articles specially when you are not aware of Push Notification concept. Also there is no default notification implementation on Android platform. So we decided to fill in the gap and allow developers to quickly run a sample android push application and recieve the push notification from third party server.

For this purpose, we created a sample application which registers the device to google service and send the registrationId to the our own third party server. The sample application displays the notification screen whenever a push notification is received by android application. You can find many server-side solutions for scripting language like php,shell script using curl etc. Since most of the enterprise applications run on J2EE platform so we decided to have a J2EE server-side solution. In our server-side implementation, you can regsiter device with registerId, view list of registerId and can send sample push to the client.

So the purpose of this article is to understand android push notification(c2dm) and start working on it quickly..Using this article developer can register the application from device and can send notification from web-interface. This way developer can also display push notification to clients or non-technical person.

There are three entity involved in android push notification:

• Android application – registers the device with C2DM server and passes the registerId to third party server
• Google’s C2DM servers – send messages to android device
• Third Party Server – send messages to C2DM server

Sample Application for Android Push Notification (C2DM):

We have taken the source code from cw-advandroid and added few tweaks in the source code. The only thing you need to do is to download the code from below link and configure it in your favorite IDE (We would prefer to use eclipse though). You need to open strings.xml file inside res/values directory and change the server URL.

<?xml version="1.0" encoding="utf-8"?>
<resources>
		<string name="app_name">PushEndpointDemo</string>
		<string name="server_url">http://192.168.1.4:8080/c2dm/notify</string>
</resources>

If you are deploying the server application (We shall discuss server-side in next section) on local machine then replace the ipaddress with your machine’s ipAddress. Now your application is ready to register your app with google and send the registrationId to your third party server.

Please notice few points before running the app on android:

• You must have android sdk 2.2+ and google service must be running on your emulator (We would recommend to install google sdk 2.3.1)
• You must register your Google account on your emulator. You can do it by going to Settings -> Accounts & Sync and add gmail account. You cannot receive push notification unless Google account is set.
• You must register your developer’s email address for your application at google . Please note one thing that this account is developer’s account and might not be account which is configured in ‘Accounts & Sync’ settings.

Third party server to register device and send Android Push Notification (C2DM):

We have built server-side solution on J2EE platform and application can be deployed on tomcat server(It might deployed fine on other servers but hav’nt tested it). Before deploying to server, you need to set the authentication code for your developer account in the application so please set it in the AUTH_KEY variable in class com.myor.c2dm.utils.Constants.java

public static final String AUTH_KEY = "" //Please set your authentication key

You can get authentication key using curl as mentioned this article or You can use the our utility com.myor.c2dm.utils.AuthenticationUtils

AuthenticationUtils.getAuthCode("<DEVELOPER_EMAIL>", "<PASSWORD>")

Please use maven to create war file and deploy it on your server web directory.

mvn clean package

You can access the application at

http://server:port/c2dm/notify

Now we ready to register and receive the android push notification on sample application. Please launch the application on your emulator or device and screen would be displayed like this:

Sample notification application

Enter your developer email account and click on the ‘Register!’ button. This shall make a call to google service to register the device for application. In the callback method, we make call to our third party server to register the registrationId for the device. We can access the registerId at

http://yourserver:port/c2dm/notify?act=1

List of devices registered on google service

Now we can send push notification to the android application by clicking the ‘Push’ link. The notification on the client–side look shall like this:

Android Push Notification Ticker text

When you drag down the notification window then full notification shall be displayed like this:

Push notification is received on device

We hope this article shall help you to power your application with Android push notification. Feel free to give us feedbacks or let us know if you have any queries.

Downloads:

• Android Client
• Third Party Server

References:

• Android Cloud To Device Messaging
• android-c2dm
• Android Cloud to Device Messaging (C2DM) – Tutorial
• Top 10 CSS Table Designs
• Demo to send notification

Caching is the vital part of any website and helps website to respond quickly. Social engine allows variety of caching solutions like file based, memcache, APC (Alternative Php cache) and Xcache for the platform. Using one of the caching solution would speed up performance of the socialengine 4. If you don’t have configured the memcached or APC or Xcache then you can use file system based caching but its much slower than the other caching solutions. We would recommend to use one of the caching solution to speed up the socialengine. In this article, we should show the instructions to install and configure caching for socialengine. We assume that the socialengine is running on the production mode.

Memcache is a distributed caching which can be used in a scenario where application is running on several web servers.All the data stored in memcached would be distributed across the all webservers.Since it requires dealing with network protocols in order to support the distributed part of caching, it is slower compared to APC/XCache cache in case of application is running on single server. While using APC/Xcache on distributed system would require manual copy of data on all the servers. APC/XCache are non distributed cache and data can be stored on the local web-server only. APC/XCache op-code caching saves on the compile and execution times of PHP thus loading of the socialengine become faster. Some people got confused whether Memcached can be used with the APC/Xcache (opcode). We can use the combination of MEMcached (as data storage) and APC/Xcache (as opcode for php).

Now we are going to show the installation and configuration of Memcached, APC and Xcache for socialengine on Centos.

Install and Configure Memcached with socialengine

1) Install memcached

  wget http://memcached.googlecode.com/files/memcached-1.4.5.tar.gz
tar zxpfv memcached*
cd memcached-1.4.5
./configure
make install
./memcached -u memcached -d -m 64 -l 127.0.0.1 -p 11211

2) Install PHP Extension.

Download and install latest stable memcache version from PECL.

wget http://pecl.php.net/get/memcache-2.2.5.tgz
tar zxvf memcache-2.2.5.tgz
cd memcache-2.2.5
phpize
./configure
make
make install

memcache.so will be install in php modules directory, now enable memcache.so extension in php.ini

To find out your php.ini location, execute following command

php -i | grep php.ini
Configuration File (php.ini) Path => /usr/local/lib
Loaded Configuration File => /usr/local/lib/php.ini

vi /usr/local/lib/php.ini

extension = “memcache.so”

save the file and restart httpd server.

/etc/init.d/httpd restart

To check is memcache extension loaded in php, execute following command.

php -i | grep memcache
memcache
memcache support => enabled

3) Configure memcached in socialengine

1. Go to http://yourserver/index.php/admin/core/settings/performance
2. Enable caching with memcached and add parameters specific to memcached.
3. Save the changes

Install and Configure APC with socialengine
By default there can be one opcode caching for php so if there is already any opcode (eaccelerator etc.) then disable that to allow APC to act as opcode.

1) Install APC

yum install php-pear
yum install php-devel
yum install httpd-devel
yum install pcre-devel
pecl install apc
echo "extension=apc.so" > /etc/php.d/apc.ini
/etc/init.d/httpd restart

To check is APC extension loaded in php, execute following command.

php phpinfo()
apc.enabled=1

2) Configure APC in socialengine

1. Go to http://yourserver/index.php/admin/core/settings/performance
2. Enable caching with APC option.
3. Save the changes

Install and Configure Xcache with socialengine

1) Install Xcache

wget http://xcache.lighttpd.net/pub/Releases/1.3.1/xcache-1.3.1.tar.gz
tar -zxvf xcache-1.3.1.tar.gz
cd xcache-1.3.1
phpize
./configure --enable-xcache
make
make install

You would need to change timelib_structs.h to install Xcache
Find line:

#include

Replace / update as follows:

#include "timelib_config.h"

Create one password for Xcache administration:

# echo -n “yourpassword”| md5sum

Then open php.ini, and add the following append configuration directives.

[xcache-common]
zend_extension      = /usr/lib/php/modules/xcache.so

[xcache.admin]
xcache.admin.user   = "admin"
;how to create md5 pwd: echo -n "password"| md5sum
xcache.admin.pass   = "a364b1d54799c713b8a377de50f8ed3c"  #easywms.com

[xcache]
; Change xcache.size to tune the size of the opcode cache
; to disable: xcache.size=0
; to enable : xcache.size=64M etc (any size > 0) and your system mmap allows
; someone said, if >=64 will get problem

xcache.size         = 32M
xcache.shm_scheme   = "mmap"
; set to cpu count (cat /proc/cpuinfo |grep -c processor)

xcache.count        = 4
; just a hash hints, you can always store count(items) > slots

xcache.slots        = 8K
; ttl of the cache item, 0=forever

xcache.ttl          = 0
; interval of gc scanning expired items, 0=no scan, other values is in seconds

xcache.gc_interval  = 0

; Change xcache.var_size to adjust the size of variable cache
; same as aboves but for variable cache, if you have not used xcache function in script,forget it

xcache.var_size     = 8M
xcache.var_count    = 1
xcache.var_slots    = 8K
xcache.var_ttl      = 0
xcache.var_maxttl   = 0
xcache.var_gc_interval =     300

xcache.test         = Off
xcache.readonly_protection = On
; for *nix, xcache.mmap_path is a file path, not directory.
; Use something like "/tmp/xcache" if you want to turn on ReadonlyProtection
; 2 group of php won’t share the same /tmp/xcache
; for win32, xcache.mmap_path=anonymous map name, not file path

xcache.mmap_path    = "/tmp/xcache"
; leave it blank(disabled) or "/tmp/phpcore/"
; make sure it’s writable by php (without checking open_basedir)

xcache.coredump_directory =   ""
; per request settings

xcache.cacher       = On
xcache.stat         = On
xcache.optimizer    = Off

[xcache.coverager]
; per request settings
; enable coverage data collecting for xcache.coveragedump_directory and
; xcache_coverager_start/stop/get/clean() functions (will hurt executing performance)

xcache.coverager    = On
; ini only settings
; make sure it’s readable (care open_basedir) by coverage viewer script
; requires xcache.coverager=On

xcache.coveragedump_directory = ""

2) Configure Xcache in socialengine

1. Go to http://yourserver/index.php/admin/core/settings/performance
2. Enable caching with Xcache option and add user’s credentials.
3. Save the changes

Conclusion
We found Xcache little faster than the Memcached/APC for socialengine 4 and used it for our client’s website.

Please feel free to comments or suggest other technique of caching.

References

• Optimizing SocialEngine 4 Performance
• How To Install memcached with memcache PHP Extension on CentOS 5.x
• Installing PHP APC on GNU/Linux Centos 5
• Install XCache for PHP 5 on CentOS
• PHP Accelerators

We looked at the various implementation of NoSQL and found solutions like Membase, Redis, CouchDB, MongoDB and SimpleDB. Since we were already using Amazon infrstructure like EC2 and S3 so we gave a shot to amazon SimpleDB. We are very pleased with the performance of simpledb as query time is below five seconds now.

Here are the instructions to use SimpleDB with Ruby on rails. We have used the simple_db gem to connect to Amazon SimpleDB web service.

How to use simple db with ruby on rails:

1. Install simple_record gem by: gem install simple_record
2. Create connection with simple db and execute sql query.

SimpleRecord.establish_connection('access_key_id','secret_access_key', :connection_mode=>:per_thread)

Company.find(:all,:conditions => ["name = ?", name],  rder => "id desc")

3. close simple db connection.It is better to use close_connection in Application controller:

after_filter :close_sdb_connection
class ApplicationController < ActionController::Base

#TO CLOSE SIMPLE DB CONNNECTION AFTER EVERY REQUEST
def close_sdb_connection
SimpleRecord.close_connection
end

There were few more requirements from client-side to generats reports from the SimpleDB. we told the client about the limitation of SimpleDB but client insist us to use SimpleDB for reporting system. We started working on the reporting and encounter some problems and limitations of SimpleDB.

Here are some major limitations :
1. It doesn’t support group by clause
2. It doesn’t support join of tables
3. It doesn’t support as (alias)
4. It can not have more than 20 comparisons in where clause.

NOTE: We cannot use full fledge ‘in’ query in the simpleDB as every value in array counted as a comparison clause.

Finally we had to revert back to mysql for reporting system.So, the moral of the story is that never use simple db blindly.
Think carefully before implementing, is it suitable or fulfill your requirement.

Here are some points to consider before implementing simple db:
** SimpleDB shifts work out of the database and onto programmers which is why the SimpleDB programming model sucks: it requires a lot more programming to do simple things. I’ll argue however that this is the kind of sulkiness programmers like. Programmers like problems they can solve with more programming. We don’t even care how twisted and inelegant the code is because we can make it work. As long as we can make it work we are happy.

References:
Amazon SimpleDB
SimpleRecord Gem
Top 10 Reasons to Avoid the SimpleDB Hype

define("URL","http://Yoursite.com:8080/action.do");

$fields = array(
  'lname'=>$last_name,
  'fname'=>$first_name,
  'title'=>$title,
  'company'=>$institution,
  'age'=>$age,
  'email'=>$email,
  'phone'=>$phone
);

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, URL);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
$output = curl_exec($ch);
curl_close($ch);

We tried to investigate the problem and our first suspicion was that the php curl is not working properly. We create a php script which just listens the request and print all the parameters in the log file.We used the above code and we were able to extract the parameters from the request. So it was the time we ask our friends google and stackoverflow. We read many articles and documentations but there were no solution for this problem. Later we realized that java server is not unmarshalling the post data(mutipart/form-data) which is in array format from php curl. So we decided to use other approach i.e. send the parameters in urlencoding string as we have http servlet on the java server side.

<?php
define("URL","http://Yoursite.com:8080/action.do");

$fields = array(
  'lname'=>$last_name,
  'fname'=>$first_name,
  'title'=>$title,
  'company'=>$institution,
  'age'=>$age,
  'email'=>$email,
  'phone'=>$phone
);

// url encoding
$fields_string = http_build_query($fields);

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, URL);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields_string);
$output = curl_exec($ch);
curl_close($ch);
?>

We hope this would help other developers to resolve this type of issue. Alternative solution to resolve this issue is to add support for multipart/form-data on java server using common’s ServletFileUpload
Please let us know your questions.

Related Posts

• Sending POST form data with php CURL
• Execute a HTTP POST Using PHP CURL

Create Easy SSL Socket Factory and create httpclient instance

import java.io.IOException;
import java.net.Socket;
import java.net.UnknownHostException;
import java.security.*;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;

import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;

import org.apache.http.conn.ssl.SSLSocketFactory;
public class EasySSLSocketFactory extends SSLSocketFactory {
        SSLContext sslContext = SSLContext.getInstance("TLS");

        public EasySSLSocketFactory(KeyStore truststore)
                        throws NoSuchAlgorithmException, KeyManagementException,
                        KeyStoreException, UnrecoverableKeyException {
                super(truststore);

                TrustManager tm = new X509TrustManager() {
                        public void checkClientTrusted(X509Certificate[] chain,
                                        String authType) throws CertificateException {
                        }

                        public void checkServerTrusted(X509Certificate[] chain,
                                        String authType) throws CertificateException {
                        }

                        public X509Certificate[] getAcceptedIssuers() {
                                return null;
                        }
                };

                sslContext.init(null, new TrustManager[] { tm }, null);
        }

        @Override
        public Socket createSocket(Socket socket, String host, int port,
                        boolean autoClose) throws IOException, UnknownHostException {
                return sslContext.getSocketFactory().createSocket(socket, host, port,
                                autoClose);
        }

        @Override
        public Socket createSocket() throws IOException {
                return sslContext.getSocketFactory().createSocket();
        }

}

We shall be using this ssl socket factory to access the secure connection to rest web service.

import java.security.KeyStore;

import org.apache.http.HttpVersion;
import org.apache.http.client.HttpClient;
import org.apache.http.conn.ClientConnectionManager;
import org.apache.http.conn.scheme.PlainSocketFactory;
import org.apache.http.conn.scheme.Scheme;
import org.apache.http.conn.scheme.SchemeRegistry;
import org.apache.http.conn.ssl.SSLSocketFactory;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager;
import org.apache.http.params.BasicHttpParams;
import org.apache.http.params.HttpParams;
import org.apache.http.params.HttpProtocolParams;
import org.apache.http.protocol.HTTP;

public class HttpUtils {
        public static HttpClient getNewHttpClient() {
            try {
                KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
                trustStore.load(null, null);

                SSLSocketFactory sf = new EasySSLSocketFactory(trustStore);
                sf.setHostnameVerifier(
                       SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);

                HttpParams params = new BasicHttpParams();
                HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
                HttpProtocolParams.setContentCharset(params, HTTP.UTF_8);

                SchemeRegistry registry = new SchemeRegistry();
                registry.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
                registry.register(new Scheme("https", sf, 443));

                ClientConnectionManager ccm = new ThreadSafeClientConnManager(params, registry);

                return new DefaultHttpClient(ccm, params);
            } catch (Exception e) {
                return new DefaultHttpClient();
            }
        }
}

Back to our original rest client code

// Create a new RestTemplate instance
				RestTemplate restTemplate = new RestTemplate();

				// The HttpComponentsClientHttpRequestFactory uses the
				// org.apache.http package to make network requests
				restTemplate
						.setRequestFactory(new HttpComponentsClientHttpRequestFactory(
								HttpUtils.getNewHttpClient()));

				// The URL for making the GET request
				final String url = "https://xxx.xxx.xxx.xxx";
                                // Initiate the HTTP GET request, expecting an array of
				// objects in response
				Object result = restTemplate.getForObject(url, Object.class);

Finally we were able to make connection over secure rest web service android application.

Feel free to comment or ask questions.

Useful resources

• Android: Trusting SSL certificates
• Disabling Certificate Validation in an HTTPS Connection
• Redmine java api

A little introduction about yii framework

Yii is a high-performance PHP framework best for developing Web 2.0 applications. Yii comes with rich features: MVC, DAO/ActiveRecord, I18N/L10N, caching, authentication and role-based access control, scaffolding, testing, etc. It can reduce your development time significantly. It has pretty good documentation and you can get more help by googling the issue. You can also compare the performance of yii framework with other popular frameworks here

A little introduction about facebook Graph API

Facebook introduced new components of Facebook Platform and one of them is Graph API. The API provides access to Facebook objects like people, photos, events etc. and the connections between them like friends, tags, shared content etc. via a uniform and consistent URI to access the representation. Every object can be accessed using the the URL https://graph.facebook.com/ID, where ID stands for the unique ID for the object in the social graph. For more details, please take a look at graph api

Sample facebook application using yii framework

We assume that you have created facebook application and have the access to the App Id and Secret Key for the application. If you have no idea about it then please take a look at the links below to get started. We also assume that you have also created the sample application using yii framework.

Configure facebook api in the application

Get php facebook api from git and place it in the vendors directory(create if not exist). We needed some additional properties so we created facebookwrapper which extends facebook api.

require 'facebook.php';
/*
 * Created on Mar 15, 2011
 *
 * Provides access to Custom application properties along with the Facebook Platform.
 *
 * @author makeurownrules
 */
 class FacebookWrapper extends Facebook{
 ...
 ...
 }

We had initialize the facebook api in the controller and added facebook filter to verify that the facebook session exist. We need to include the wrapper in the main controller in the components package.

Yii::import('application.vendors.*');
require_once('facebook-php-sdk-b14edfa/src/facebookwrapper.php');
...
...
function __construct($id,$module=null) {
    parent::__construct($id, $module);
    // Get an instance of the Facebook class distributed as the PHP SDK by facebook:
    Yii::app()->params['facebook'] = new FacebookWrapper(array(
      'appId'  => 'appId',
      'secret' => 'secretKey',
      'cookie' => true,
      'appName' => 'App Name',
      'canvasPage' => 'http://apps.facebook.com/appname/',
      'canvasUrl'  => 'http://example.com/appname/'
      ));

  }

  /**
   * Filters for all the request coming to facebook application
   */
  public function filters() {
    return array(
            'facebook',
    );
  }

  public function filterFacebook($filterChain) {
    header('P3P: CP="CAO PSA OUR"');
    $facebook = Yii::app()->params['facebook'];
    $session = $facebook->getSession();
    $fbme = null;
    if ($session) {
      try {
        $uid = $facebook->getUser();
        $fbme = $facebook->api('/me');
      } catch (FacebookApiException $e) {
         //handle error
      }
    }
    if ($fbme) {
      // call $filterChain->run() to continue filtering and action execution
      $filterChain->run();
    }
    else {
      $appId = $facebook->getAppId();
      $canvasUrl = $facebook->getCanvasPage();
      // Set the required permissions for the application
      $perms = "publish_stream,friends_birthday, offline_access, email";
      $loginUrl = "https://www.facebook.com/dialog/oauth?scope=".$perms.
        "&client_id=".$facebook->getAppId().
        "&redirect_uri=".urlencode($canvasUrl);
      location =  'top.location.href=';
     echo("");
      Yii::app()->end();
    }
  }

After initializing the facebook API, we can access facebook instance in the SiteController or any other controller.

public function actionIndex()
	{
		$facebook = Yii :: app()->params['facebook'];
                $myuid = $facebook->getUser();
	  // renders the view file 'protected/views/site/index.php'
		// using the default layout 'protected/views/layouts/main.php'
		$this->render('index',array('myuid'=> $myuid));
	}

So facebook api can be accessed and used to publish on wall, send message or change the status of user.

Source code and Demo

Source for sample application is available here.
Demo is available here

References

We highly recommend to go through basic facebook application documentation.

Official Facebook documentation / developer site:
Facebook Developers Site

Facebook tutorials:
publish post with media on wall using facebook Graph API

Facebook PHP Tutorial
Developing with the Facebook Platform and PHP

1. Using Javascript SDK
2. Using Graph API

We can use facebook’s javascript api to publish media of friend’s wall like :

// Include facebook javascript by including http://connect.facebook.net/en_US/all.js
// Initialize facebook session
FB.init({
		appId  : ' getAppId();?>', // application Id
		status : true, // check login status
		cookie : true, // enable cookies to allow the server to access the session
		xfbml  : true  // parse XFBML
	});
function publishStream(targetId, name, link, picture, source, userName,message) {
FB.ui(
        {
        	 method: 'feed',
        	 to: targetId,
                 name: name,
                 link: link,
                 picture: picture,
                 source:  source, // The URL of a media file (e.g., a SWF or video file) attached to this post
                 caption: 'Shared by '+userName,
                 actions:  {name: 'Try App', link: link},
                 message: message

        },
        function(response){
		     if (response && response.post_id) {
		      alert('Post was published.');
		     } else {
		       alert('Post was not published.');
		     }
        });
    }

Second way is to use Graph API:

http://www.facebook.com/dialog/feed?app_id=app_id&link=link&picture=picture &caption=caption&name=name&source=source &message=message&redirect_uri=response

We are assume that you have already worked on facebook api. We would like to answer your questions.

Conversion of file data to attachment_fu format

Using attachment_fu plugin we were expecting the file data in params[:ASSET_DATA] but with swfupload data was coming as params[:Filedata]. So we wrapped the Filedata from swfupload and put that in the desired format

 data = {"uploaded_data" =>params[:Filedata]}

Setting mimetype of the file

Attachment_fu requires the mimetype to be set correctly before it will accept the files. Matt Aimonetti wrote mimetype-fu to help attachment_fu properly set the mimetype.

We override the uploaded_data of attachment_fu in our model object.

 #override attachment_fu plugin
  def uploaded_data=(file_data)
    return nil if file_data.nil? || file_data.size == 0
    self.filename = file_data.original_filename if respond_to?(:filename)
    extension = file_data.original_filename.slice(/\.\w+$/)
    # Map file extensions to mime types.  Thanks to bug in Flash 8
    # the content type is always set to application/octet-stream.
    mime = MIME::Types.type_for(self.filename)[0]
    self.content_type = mime.blank? ? file_data.content_type : mime.content_type
    #Custom name of the file
    self.filename  = "xxxx_" + Digest::SHA1.hexdigest(Time.now.to_s) + extension
    if file_data.is_a?(StringIO)
      file_data.rewind
      self.temp_data = file_data.read
    else
      self.temp_path = file_data.path
    end
     #callback after attachment is uploaded
    after_attachment_uploaded
  end

Please let us know if you have any issues using above code or have any questions.

References

• SWFUpload
• Ruby on Rails
• Uploading Files using SWFUpload

We override buildprompt function from the validationengine and added functionality to pick the customized error message.

var buildPrompt = $.validationEngine.buildPrompt;
$.validationEngine.buildPrompt = function(caller, promptText, type, ajaxed) {
  // Get the rules to map the message for a method
  var rulesRegExp = /\[(.*)\]/;
  var getRules = rulesRegExp.exec($(caller).attr('class'));
  var str = getRules[1];
  var pattern = /\[|,|\]/;
  var rules = str.split(pattern);
  //Check if title attribute present in the element
  //otherwise we shall use default error message
  if ($(caller).attr('title')) {
    var getMessages = rulesRegExp.exec($(caller).attr('title'));
    var str = getMessages[1];
    var pattern = /\[|,|\]/;
    var messages = str.split(pattern);

    var j = 0;
    newPrompt = "";
    for ( var i = 0; i < rules.length; i++) {
     rules = $.validationEngine.settings.allrules[rules[i]]
      if (rules) {
        if (promptText.indexOf(rules.alertText) != -1) {

          newPrompt += "

” + messages[j] + ” “; } j++; } } promptText = newPrompt; } buildPrompt(caller, promptText, type, ajaxed); }

So in the plugin, we have added error messages in the ‘title’ attribute and this gives the flexibility to customize the error message for different field. So here is the example where custom error message can be added:

Now validation engine would display custom error message as shown below:

Please feel free to add your comments/questions or Suggestions

In this Artcle, I used geonames.org Rest service to fetch the near places for a specific postal code which in radius.So first of we need to configure the project with maven. Please refer this to get the details of configuring maven for Android.

Your pom.xml file should look like this:

    src
    ${project.artifactId}

        com.jayway.maven.plugins.android.generation2
        maven-android-plugin
        ${maven-android-plugin-version}

${android.home}
${android-platform}

            Android2.1

          true
          true

        true

        maven-compiler-plugin
        ${maven-compiler-plugin-version}

After configuring the android project with maven, we are ready for some action. We need to connect to geonames.org to get the data from the rest service. Using spring platform for mobile, its very easy to do that. So here is the code which connects to service and fetch the results:

  // Create a new RestTemplate instance
  RestTemplate restTemplate = new RestTemplate();

  // The HttpComponentsClientHttpRequestFactory uses the
  // org.apache.http package to make network requests
  restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestFactory());

  // The URL for making the GET request
  final String url = "http://ws.geonames.org/findNearbyPostalCodesJSON?postalcode={postalCode}&country={countryCode}&radius={radius}";

  // Initiate the HTTP GET request, expecting an array of State
  // objects in response
  PostalCodes result = restTemplate.getForObject(url,
      PostalCodes.class, this.postalCode, this.countryCode,
      this.radius);

In case restful service uses digest authentication then you can use this:

DefaultHttpClient client = new DefaultHttpClient();
Credentials defaultcreds = new UsernamePasswordCredentials("xxxx", "xxxx");

client.getCredentialsProvider()
  .setCredentials(new AuthScope("xxxx", 80, AuthScope.ANY_REALM), defaultcreds);
// The HttpComponentsClientHttpRequestFactory uses the
// org.apache.http package to make network requests
restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestFactory(client));

The response from service returns json objects so we require pojo objects to populate the response and use them in android client. So we need to create two Pojo as

public class PostalCodes {
	private PostalCode[] postalCodes;
..
}
public class PostalCode {
	private String adminName2;
	private String adminCode2;
	private String adminCode1;
	private String postalCode;
	private String countryCode;
	private Double lng;
	private String placeName;
	private Double lat;
	private String adminName1;
...
}

So all code is in place and its time to build the android project using:

mvn clean install

We can start the emulator using

mvn android:emulator-start

We can deploy the application using

 mvn android:deploy

Home Screen for the android client would be:

Results from Restful service:

So we have created restful client for android without much efforts. Please feel free to send your comments/questions.

You can find the source code here

References:

Spring-AndroidAndroidgeonamesAndroid Development with Maven

Image Crop:

There are many javascripts from hotscripts which can be used for image cropping and I have used one of the javasript “Javascript Image Cropper UI” .

Image Processing

The above javascript shall give us the dimensions for the cropped image and we need to process the image using gem ‘RMagick’

=start rdoc
This method takes crop image dimensions along with the original image path
and re-sized the image to the desired size
x: left most x coordinate of image for cropped image from JavaScript
y: left most y coordinate of image for cropped image from JavaScript
Width: width of image for cropped image from JavaScript
Height: height of image for cropped image from JavaScript
Path: Image path from JavaScript
=end

def self.update_attributes_crop(x,y,width,height,path)

    # Get the path of the Directory where Image should be stored
    path = File.dirname(__FILE__) + "/../../public/#{path}"

    orig_img = Magick::ImageList.new(path)

    scale = orig_img.crop(x.to_i,y.to_i,width.to_i,height.to_i, true)

    crop_img = Magick::Image.new(width.to_i,height.to_i)

    orig_img = crop_img.composite(scale,0,0,Magick::OverCompositeOp)

    orig_img.write(path)

end

Upload image to Amazon S3
If you want to upload the image to Amazon S3 then you can use :
S3_copy(source, destination, bucket) line of code in your controller

=start rdoc
This method upload the file from local drive to Amazon S3 bucket
Source: source(path) of image from rails server
Destination: destination (relative path) of Image on S3 server
Bucket: your S3 bucket name for image upload
=end
def S3_copy(source, destination, bucket)

	AWS::S3::Base.establish_connection!(

	:access_key_id => S3_access_key_id,

	:secret_access_key => S3_secret_access_key

	)

	AWS::S3::S3Object.copy(source,destination,bucket,:access => :public_read)

	policy = AWS::S3::S3Object.acl(destination,bucket)
	policy.grants << AWS::S3::ACL::Grant.grant(:public_read)
	policy.grants << AWS::S3::ACL::Grant.grant(:public_read_acp)

	begin

		AWS::S3::S3Object.acl(destination, bucket, policy)

	rescue

		p 'Issue occurred while granting access for the file'

	end

end

I hope using above approach you can do image cropping and uploading. Please let me know if you have any questions

I have to change mysql configuration file( my.cnf) as:

1. tmp_table_size: from 32M  to 128M (This might not work because of known bug)
2. max_heap_table_size: 32M to 128M

After this change, query is taking around 10 seconds for execution which is quite faster.

Reason:

Mysql creates temporary table for  the results is an on-disk table.If an internal temporary table is created initially as an in-memory table but becomes too large, MySQL automatically converts it to an on-disk table. The maximum size for in-memory temporary tables is the minimum of the tmp_table_size and max_heap_table_size values.The max_heap_table_size system variable determines how large the table is permitted to grow and there is no conversion to on-disk format.

If the value of max_heap_table_size is small then converts in-memory table to an on-disk table.Due to this on-disk table, it would take too much time for I/O read in comparison of  in-memory table(temp table).

Backup MySQL Database

$mysqldump -u {username} -p{password} {database} | gzip -9 > [database.sql.gz]

We can use scp utility to transfer this database dump to new server. if you are not aware of scp then please google and you shall find lots of information about the usage

We can easy extract the data by using gzip

$gzip database.sql.gz

Restore MySQL Database
This part is tricky as mysql dump size grow. A simple command for restoring database is

$mysql -u {username} -p{password} {database} < database.sql

or use this in case you have compressed file

gunzip < {database.sql.gz} |mysql -u {username} -p{password} {database}

This should have done the magic of restoring the database but NO for larger database. For restoring large database, you must change configurations file my.cnf or specify the –max_allow_packet =1G during restore command.

[mysqld] section
max_allowed_packet= 1G

Please note that mysql allows up to 1G for max_allowed_packet.

If your packet size(not database size) is more than 1G then consider –extended-insert off during database dump. It shall take more time for database restore but it would work.

I would like to see your comments on this.

I wanted to speed up the mobile website so I decided to do following:

• Minify the CSS and JS files
• Gzip Compression of minified files
• Uploading the Static Contents on Amazon S3

I wanted to use Capistrano script to automate the process but found no straight forward solution. So I decided that I build the solution and Share it with you guys .

Pre-Requisites:
Following are the list of plugins required on your host machine

• Capistrano 2.0.0 or greater
• YUI Compressor 2.4.2 or greater (For this you need java)
• AWS-S3 Gem
• Capistrano plugin extension S3 Asset Host Syncher
• gzip

Installation
First we need to set up Capistrano for our project and generate a capFile for the deployment. We also need to install supporting gems and libraries. Type following from the Root of your project:

$> sudo gem install capistrano
$> sudo gem install aws-s3
$>ruby script/plugin install git://github.com/spatten/synch_s3_asset_host.git (Make sure its installed in /vendor/plugins/synch_s3_asset_host otherwise download the plugin from site).
$>wget http://yuilibrary.com/downloads/download.php?file=2a526a9aedfe2affceed1e1c3f9c0579 (Extract the yuicompressor-xxx.jar and put it in /vendor/plugins/yuicompressor directory).

Capistrano configuration

In RAILS_ROOT/config/deploy.rb

Specify one of your web hosts as an “asset_host_syncher”. If you only have one web host, you don’t make a new line for this,
just edit the existing line that sets your :web role
role :web, webserver1, :asset_host_syncher => true

Create a new task minify in the cap file:

task :minify do
  compressor = "java -jar #{current_path}/vendor/plugins/yuocompressor/yuicompressor-2.4.2.jar"
  #name of the javascript files to be minify and compressed
   js_list = ["xxxx","yyyyy"]
   javascript_path = #{current_path}/public/javascripts
   run  "cd #{javascript_path}"
   js_list.each do |js|
     run "#{compressor} --type js #{js}.js -o #{js}.min.js"
     run "gzip -c #{js}.min.js >#{js}.min.js.gz"
  end

  #name of the style files to be minify and compressed
  css_list = ["xxxx","yyyy"]
  css_path = #{current_path}/public/stylesheets
  run ("cd #{css_path}")
  css_list.each do |css|
     run "#{compressor} --type css #{css}.css -o #{css}.min.css"
     run ("gzip -c #{css}.min.css >#{js}.min.css.gz"
  end
  deploy::s3_asset_host:synch_public
end

Configure Capistrano to minify before doing the final symlink task:

before "deploy:symlink", "minify"

S3 configuration

Create a file in RAILS_ROOT/config called synch_s3_asset_host.yml. Add the following to it,
and edit to suit:

AWS_ACCESS_KEY_ID: ‘your access key here’

AWS_SECRET_ACCESS_KEY: ‘your secret key here’

asset_host_name: “xyz.com”

Configure Single asset host

For a single asset host, simply add the following line to RAILS_ROOT/config/environments/production.rb:

config.action_controller.asset_host = "http://xyz.com"

Some tweaks in S3synch plugin
As you know that Amazon S3 have its own ways to use compressed css/js so we need few tweaks in the S3 Asset Host Syncher.To make it work gzip compression, I had to add following in the s3sync/s3sync.rb (Line 507)

if @path.include? '.gz'
  headers['Content-Encoding'] = 'gzip'
end

Now we are ready to deploy our stuff on site and improve the performance of the site.

Deploy

$> cap s3_asset_host:setup (No need if you have bucket already)
$> cap deploy

We can add some more tweak like excluding directories for synching to Amazon S3 bucket and I have added that tweak in the comments of Asset Syncher. If there are better solutions, let me know. I’d love a simpler solution for this.