Opensource tools for Analyzing Website Traffic Stats

While developing reports fr one of my applications I have got some interest in data and analytic - while googling I have got some of the good works that are open sourced :) [There are always some good folks online :B] . I am listing couple of the tools here:

Hummingbird:

"Hummingbird lets you see how visitors are interacting with your website in real time."

Hummingbird is built on top of Node.js, and utilizes Websocket to provide 'Live' stats.

Platform : Node.js and javascript





Pikwik

"Piwik is the leading open source web analytics platform that gives you valuable insights into your website’s visitors, your marketing campaigns and much more, so you can optimize your strategy and online experience of your visitors."

Platform: PHP- MySql



For more of them check the list in the following link :

http://en.wikipedia.org/wiki/List_of_web_analytics_software

Opensource applications with CodeIgniter

So far I have a crash for CodeIgnier and whenever I start developing for myself my first preference is always been CI :)
In this post I will be listing some of the Opensource projects built on top of CI :

Blogging tool :
Open blog: http://www.open-blog.info/
Stblog: https://github.com/stblog/Stblog >
Simple blogging App: https://github.com/indranil/scribe
Blog system App: https://github.com/mengu/ciblog
Blogging App: https://github.com/brenden/Ignited-Blog

CMS:
Ionize Cms: http://www.ionizecms.com/
Egypt CMS: http://blazeeboy.github.com/Codeigniter-Egypt/
Max CMS: http://max-3000.com/
Getfuel Cms: http://www.getfuelcms.com/
CodeFight Cms: http://codefight.org/
Pyro Cms: http://pyrocms.com/
Image CMS: https://github.com/firstrow/ImageCMS
Manager CMS: https://github.com/souzadavi/CMS
Photo gall CMS: https://github.com/bensonarts/GalleryCMS
Fotochest gall CMS: http://fotochest.com/

Forum

Forum App: http://sourceforge.net/projects/forumprojesi/
Dove Forum: https://github.com/ChrisBaines/Dove-Forums

Social Networking
Social Network Framework: https://github.com/valfreixo/Social-Network-Framework
Book Sharing social network: https://github.com/viaduk/BookSharing
Maran Cake Melody - Social Portal using Cakephp or CodeIgniter + Doctrine ORM : http://code.google.com/p/marancakemelody/

E-commerce
Social ecommerce platform: https://github.com/lammoth/Cloubiz

Other:
Classroom Bookings App: http://classroombookings.com/
Bamboo Invoice App: http://www.bambooinvoice.org/
MyClientBase Invoice App: http://myclientbase.com/
68 Knowledge Base App: https://github.com/68kb/
Sitemanager App: http://designelemental.net/sitemanagr
QuickSnaps Gall App: https://github.com/eoinmcg/QuickSnaps

ToDo list App: https://github.com/Ajmal/todo

File Hosting App: http://net.tutsplus.com/tutorials/php/creating-a-file-hosting-site-with-codeigniter/

Pivot Tables

For some reporting started working on some pivot tables. To me these tables are one of those initial rich data visualization formats.
What is it ? :
"In SQL, a pivot table is a set of data that is transformed from a collection of separate rows to a collection of columns. In relational databases, such as Microsoft SQL Server, Oracle and MySQL, pivot tables can be used to simplify extensive data in order to make it easier to read and understand. To create a pivot table, an aggregate is used against a set of data to distribute multiple rows of a single column into a single row with multiple columns. This essentially pivots the result set sideways."

How to :

-Pivot by Aggregated Case Statement

By using an aggregate function (SUM, AVG, MIN, MAX) around a case statement in a SQL query, we are able to achieve the same result as the PIVOT function with less work.

- Sample data: To better understand a pivot table, an example of some sales data is listed here. Copy the following into Microsoft SQL Server Management Studio to try out the examples.

Create table PivotTestTable
(CustName varchar(8),
Item_Type varchar(8),
Item_Amount numeric(6,2))

insert into #PivotTestTable
select 'Jason', 'Computer', 435.34
union
select 'Jason', 'Software', 243.54
union
select 'Jason', 'Monitor', 158.23
union
select 'Alison', 'Computer', 345.89
union
select 'Alison', 'Software', 78.78
union
select 'Alison', 'Monitor', 123.45

- UnPivoted data : When the temp table, #PivotTestTable, is queried, the result is the following.

CustName Item_Type Item_Amount
-------- --------- -----------
Alison Computer 345.89Alison Monitor 123.45
Alison Software 78.78Jason Computer 435.34
Jason Monitor 158.23Jason Software 243.54

As you can see, the result set shows two customers, Alison and Jason, who have purchased three different types of items. There are six rows of data for two customers. If we wanted to see the data in a single row per customer, we would use a pivot table to achieve the desired result.

- Pivoted data : Try the following query and see how the output looks now:

SELECT
CustName as Total_Sales_By_Cust,
sum(case Item_Type when 'Computer' then Item_Amount end) as Computer,
sum(case Item_Type when 'Monitor' then Item_Amount end) as Monitor,
sum(case Item_Type when 'Software' then Item_Amount end) as Software
FROM PivotTestTable
GROUP BY CustName

Or the same can be done as :

SELECT
CustName as Total_Sales_By_Cust,
sum(Item_Type = 'Computer') as Computer,
sum(Item_Type = 'Monitor' ) as Monitor,
sum(Item_Type ='Software' ) as Software
FROM PivotTestTable
GROUP BY CustName

Or you can even COUNT instead of SUM

Currently I am only playing with the SQL to generate the pivot tables; But yeah, there are couple of JS library that you can utilize to play with the Pivot tables on the fly on the client :) :

1. Pivot.js
2. jQuery.pivot

Reference:

• http://www.ehow.com/about_5336679_pivot-table-sql.html
• http://dev.mysql.com/tech-resources/articles/wizard/print_version.html
• http://forums.mysql.com/read.php?52,527105,527163

PHP SOAP client to consume JAX-WS

For couple of days I was googling and researching for a descent solution to my problem to access and consume the services exposed over a JAX-WS from my php application. Actually, I have to write a client so that I can save data from my php application (A subscriber application info management system for the telco company I am working for) to IN database where I have to do so over a JAX-WS interface.
Usually, I woud've followed my previous post. But it din work in this case- the problem and a basic setup is I got from the following location:
http://www.lampjunkie.com/2010/03/get-phps-soapclient-to-speak-with-javas-jax-ws/
But that either din resolve my problem entirely; as my JAX-WS needs a BASIC HTTP AUTHENTICATION to access it. So, following was my solution :

Note:
From php manual I must quote that you must set uri and location directives while invoking webservice with wsdl.I mean when you create the SoapClient object.

Now, that is how I retrieved data from the web service. Next, I will try to save the data and will share how I do it.

Reference:

1. php manual for soap client. A must read with care of the all the options.
2. soapUI. A must have tool while working with soap - to easy analyze things.
3. wsf/php. A framework for php to easily play with enterprise level web services. I will write my experience with it some other time.

My Instant Continuous Integration environment for development

A simplest dev environment for continuous integration for a dev team [considering a php project with mysql db] :
1. Install Visual SVN server - its for windows and easy to install and manage SVN server.
2. Create a post commit hook for the repo; to do that - In the VisualSVN Server GUI Tool:
-Select your repository
-Right click Properties Select the Hooks tab
-Select the Post-commit hook
-Click the Edit button.
-Enter a line like this into the textbox:

c:\program files\visualsvn server\bin\svn.exe" export https://svn.yourserver.com:4433/svn/YourRepository/Websites/YourWebsite/ C:\inetpub\wwwroot\YourWebsite --quiet --non-interactive --force --username youruser --password yourpassword

-Click Ok.
3. Now for the DB Sync follow the steps in the following article : [However, you might need some modification to meet your need]
http://www.decodephp.com/2008/09/29/svn-and-mysql-schema-synchronization/

Surely, You can add more flavor to this environment, like you can add a script to run each day so that it grabs all the commit logs and emails to the test team and/or stack holders or may to the entire team - so that everybody remains better informed.

Hope this will help us a to remain synced in an Agile team.
Those who is not aware of the term Continuous Integration, you can check this wikipedia link

Installing php driver for MongoDB

      &   


This is the post where I will configure php to work with mongoDB. On my previous post I have put the teps I followed to install mongoDB on my Ubuntu [TurnKey Linux] running on VirtualBox.
1.Run:

sudo pecl install mongo

2.Open your php.ini file and add to it:

extension=mongo.so

It is recommended to add this to the section with the other "extensions", but it will work from anywhere within the php.ini file.
3.Restart your web server (Apache, nginx, etc.) for the change to take effect.
Note:
pecl requires that pear be installed. For those using apt-get, you may need to run the following:

sudo apt-get install php5-dev php5-cli php-pear

However, you can download the source for the driver and can compile it manually :

shell> tar -xzvf mongo-1.2.6.tar.gz
shell> cd mongo-1.2.6
shell> phpize
shell> ./configure
shell> make
shell> make install

Check with the phpinfo

Starting MongoDB and php on Ubuntu

Installing MongoDB on Ubuntu:

1. Add 10gen package to source.list 
The 10gen package contains the latest mongoDB version, append below line to the end of the file “/etc/apt/sources.list”

deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen

For example, you can vim the “source.list” and append the 10gen package like this :

$ sudo vim /etc/apt/sources.list

File : /etc/apt/sources.list

#...content omitted ...
## This software is not part of Ubuntu, but is offered by third-party
## developers who want to ship their latest software.
deb http://extras.ubuntu.com/ubuntu natty main
deb-src http://extras.ubuntu.com/ubuntu natty main
 #mongo repo ##########
deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen

 2. Update package
Update the modified “/etc/apt/sources.list” :

sudo apt-get update

Now, a new “mongodb-10gen – An object/document-oriented database” is available for install.

  3. Add GPG Key

10gen package required GPG key, import it :

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

   4. Install mongodb-10gen

Everything is ready, now you can Install the mongoDB package :

sudo apt-get install mongodb-10gen

    5. Post-InstallationVerification

Now, mongoDB is installed, started, and auto start mongoDB script is generated to “/etc/init/mongo” and “/etc/init.d/mongo“. In addition, all mongoDB files are copied to “/usr/bin” folder.

The main configuration file “mongodb.conf” is located at “/etc/mongodb.conf“, change the values to customize your mongoDB server.

File : mongodb.conf

# mongodb.conf
# Where to store the data.
# Note: if you run mongodb as a non-root user (recommended) you may
# need to create and set permissions for this directory manually,
# e.g., if the parent directory isn't mutable by the mongodb user.
dbpath=/var/lib/mongodb
#where to log
logpath=/var/log/mongodb/mongodb.log logappend=true
#port = 27017
#......

   6. Verification

To verify it, just connect it with “mongo”

$ mongo
MongoDB shell version: 1.8.1 connecting to: test
>

on my next post I will try to connect php with this mongoDB.

Parsing Email with PHP

I was working for a Email Advertisement tool,recently. And the first thing I was put to work with is to develop a parser that parses and incoming email and decides whether it is a bounced back email. After couple of googling I got some fare enough solution that I wanted to use for the solution. The steps I followed for the solution are adopted from the following :
HOWTO: Email Bounce handling with PHP, Exim & CPanel, Catch bounces from non-deliverable emails

And the very basic parser I wrote can be downloaded from the following location:
Download the email parser for bounce management
[Bascially It's a codeIgniter controller, but you can use it as you wanna use it]

However, I had to go some more details parsing, later to decide the type of bounce or the reason for the bounce.First I tried parsing the log file for the exim [exim_mainlog file] and then parsing the entire body of the bounced back emails. I will share two more posts on them in this series.

Installing ffmpeg and other extensions and tools to use with php

Installing FFMpeg

yum install ffmpeg ffmpeg-devel

If you get package not found, then you will need to add few lines in the yum repository for dag packages installation. Create a file named dag.repo in /etc/yum.repos.d with the following contents on it

[dag]
name=Dag RPM Repository for Red Hat Enterprise Linux
baseurl=http://apt.sw.be/redhat/el$releasever/en/$basearch/dag
gpgcheck=1
enabled=1

then

yum install ffmpeg ffmpeg-devel

If everything is fine, then the installation should proceed smoothly. If not you will get something like warning GPG public key missing .

Common Errors

To fix rpmforge GPG key warning:

rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm

For more information refer to this faq depending on Centos version
Missing Dependency Error:
If you get missing dependency error like shown below, in the middle of ffmpeg installation

Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package ffmpeg
Error: Missing Dependency: libtheora.so.0(libtheora.so.1.0) is needed by package ffmpeg
Error: Missing Dependency: rtld(GNU_HASH) is needed by package ffmpeg
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package imlib2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package a52dec
Error: Missing Dependency: rtld(GNU_HASH) is needed by package imlib2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package gsm
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package x264
Error: Missing Dependency: rtld(GNU_HASH) is needed by package xvidcore
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package lame
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package a52dec
Error: Missing Dependency: rtld(GNU_HASH) is needed by package faad2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package x264
Error: Missing Dependency: rtld(GNU_HASH) is needed by package lame
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package xvidcore
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package faac
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package faad2
Error: Missing Dependency: libgif.so.4 is needed by package imlib2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package faac
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package gsm
Error: Missing Dependency: libpng12.so.0(PNG12_0) is needed by package imlib2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package libmp4v2
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package libmp4v2

then most commonly you have GLIB 2.3 installed instead of GLIB 2.4 version. To check the current GLIB version installed on your server. just use

yum list glib*

and it should list the latest GLIB package version.

The reason i was getting this error was my rpmforge packages was pointed to centos 5 versions instead of centos 4.6.


To fix dependency error:

To fix this error, you might need to check your rpmforge packages compatible to the release of your existing CentOS version.
Check the file /etc/yum.repos.d/rpmforge.repo and it should look like for Centos 4.6(Final). If you have lines like http://apt.sw.be/redhat/el5/en/mirrors-rpmforge you might need to make changes to therpmforge.repos like shown below

Note: Backup the original rpmforge.repo file before you edit its content.

[rpmforge]
name = Red Hat Enterprise $releasever - RPMforge.net - dag
#baseurl = http://apt.sw.be/redhat/el4/en/$basearch/dag
mirrorlist = http://apt.sw.be/redhat/el4/en/mirrors-rpmforge
#mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge
enabled = 1
protect = 0
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag
gpgcheck = 1

To know what linux type and version you are running

cat /etc/redhat-release

Once this is done, do again yum install ffmpeg.
This trick resolved the problem in my linux box running Centos 4.6 and this is the only way i found to install ffmpeg using yum.
To check the FFmpeg working:

Finally, check the ffmpeg whether it is working or not.

ffmpeg
ffmpeg -formats
ffmpeg --help

// This lists path of mpeg, its modules and other path information

ffmpeg -i Input.file Output.file

To check what audi/video formats are supported

ffmpeg -formats > ffmpeg-format.txt

Open the ffmpeg-formats.txt to see the output
D means decode
E means encode
V means video
A means audio
T = Truncated


Install FFMPEG-PHP Extension
FFmpeg-php is a very good extension and wrapper for PHP which can pull useful information about video through API interface. Inorder to install it you will need to download the source file and then compile and install extension in your server. You can download the source tarball : http://ffmpeg-php.sourceforge.net/

wget /path/to/this/file/ffmpeg-php-0.6.0.tbz2

tar -xjf ffmpeg-0.6.0.tbz2

phpize
./configure
make
make install

Common Errors

1. If you get command not found error for phpize, then you will need to do yum install php-devel
2. If you get error like "ffmpeg headers not found" while configuring the source.
configure: error: ffmpeg headers not found. Make sure ffmpeg is compiled as shared libraries using the --enable-shared option
then it means you have not installed ffmpeg-devel packages.
To Fix: Just install ffmpeg-devel using

yum install ffmpeg-devel

3. If you get an error like shared libraries not found problem and the program halts in the middle, then you must specify the ffmpeg installed path explicitly to the ./configure.

configure: error: ffmpeg shared libraries not found. Make sure ffmpeg is compiled as shared libraries using the --enable-shared option

To Fix:
1. First find out the ffmpeg path with ffmpeg --help command. The prefix default path should be like /usr/local/cpffmpeg
2. Configure the FFmpeg-php with --with-ffmpeg option

./configure --with-ffmpeg=/usr/local/cpffmpeg

That should resolve the problem!


Issue that I faced while installing the ffmpeg-php of version 0.6.0 :

/tmp/ffmpeg-php-0.6.0/ffmpeg_frame.c: In function ‘zim_ffmpeg_frame_toGDImage’:
/tmp/ffmpeg-php-0.6.0/ffmpeg_frame.c:336: error: ‘PIX_FMT_RGBA32’ undeclared (first use in this function)
/tmp/ffmpeg-php-0.6.0/ffmpeg_frame.c:336: error: (Each undeclared identifier is reported only once
/tmp/ffmpeg-php-0.6.0/ffmpeg_frame.c:336: error: for each function it appears in.)
/tmp/ffmpeg-php-0.6.0/ffmpeg_frame.c: In function ‘zim_ffmpeg_frame_ffmpeg_frame’:
/tmp/ffmpeg-php-0.6.0/ffmpeg_frame.c:421: error: ‘PIX_FMT_RGBA32’ undeclared (first use in this function)
make: *** [ffmpeg_frame.lo] Erreur 1

Fix to this issue :

Just open the ffmpeg_frame.c file and replace the PIX_FMT_RGBA32 with PIX_FMT_RGB32. With vi you can do it as :

vi ffmpeg_frame.c
:%s/PIX_FMT_RGBA32/PIX_FMT_RGB32
:w

However I just opened it on a text editor on my windows and applied find and replace ;)

After the replacement is done you have surely you have to do the configure and make and make install as :

phpize
./configure
make
make install

[before performing a make this time it is recommended you do a make clean]

Editing php.ini
Once you have done that without any problems then you will see the php extension file /usr/lib/php/modules/ffmpeg.so and you will need mention that extension in php.ini file

vi /etc//php.ini

Put the below two lines at the end of the php.ini file

[ffmpeg]
extension=ffmpeg.so

Then restart the server service httpd restart. In my case I had a server from godaddy and I had to do the following to restart the webserver :

/usr/sbin/apachectl graceful

To check whether ffmpeg enabled with php, point your browser to test.php file. It should show the confirmation of installed ffmpeg php extension

// #test.php
          phpinfo()
?>

This is what you are suppose to get :

If any case the ffmpeg does not show in the phpinfo() test make sure that php.ini path to ffmpeg.so is correct. Still the problem occurs, the reason could be you might be using older versions of ffmpeg-php which is buggy. Just download the latest version of ffmpeg-php source then compile it.

Installing Mplayer + Mencoder
To know more about Mplayer
Just issue the following yum commands to install the rest of the packages.

yum install mplayer mencoder

Installing FlvTool2
Flvtool2 is a flash video file manipulation tool. It can calculate metadata and can cut and edit cue points for flv files.
If you are on Centos 5 try yum install flvtool2 [worked fine with me] with dag repository and if you get package not found you will need to manually download and compile the flvtool2. You can download latest version of flvtool2 here: http://rubyforge.org/projects/flvtool2/

wget
ruby setup.rb config
ruby setup.rb setup
sudo ruby setup.rb install

If you get command not found error, it probably means that you dont have ruby installed.

yum install ruby

Thats it! Once ffmpeg works fine with php extension, download a sample video, convert to .flv format in the command line and plug it to flowplayer to see it work on your web browser. Try also to download the video file offline and see whether the converted flv file works well with both audio and video.
Now following is how you might wanna play with this beautiful tool/toolset :
This is how ffmpeg actually works :

Here is a some tutorials to play with ffmepg.

http://www.phpro.org/tutorials/Video-Conversion-With-FFMPEG.html

Reference :

• http://adminsdiary.com/linux/ffmpeg-and-related-modules-installation

Running CodeIgniter from shell on Cronjobs

I am actually tuklifying from the CodeIgniter wiki - just adding my experience - things I faced and solution to the issue.
Steps :
1.Make a directory for your cron scripts. I just used scripts and put it in the same directory that my CI system folder is in.
2. Copy the index.php file from your DOCUMENT_ROOT (the script that normally initiates a connection to CI from a web browser) to the scripts directory, and rename it to something meaningful, such as mailsender.php.
3. Add the following line at the very first line of the mailsender.php:

#!/usr/local/bin/php -q

4. Edit this file and make some slight adjustments. First, we don’t want the script to timeout during execution, so somewhere toward the top we add:

set_time_limit(0);

5. Set the path to the system folder, if it isn’t correct:

$system_folder = "../system";

6. We need to let CI know what controller/method we want to access, and any other URI variables we need. Normally this happens in the URL, so we emulate it by setting PATH_INFO manually:

$_SERVER['PATH_INFO'] = '/mailprocessor/sendmail';

$_GET = ''; // Required for some installations

However, if the config uri protocol is set to REQUEST_URI, then CI doesn’t use PATH_INFO, so instead, emulate REQUEST_URI manually:

$_SERVER['REQUEST_URI'] = '/mailprocessor/sendmail';

7. You create a mailprocessor.php controller (in the application/controllers dir) with a sendmail method, and make it do what it needs to do, load libraries/models/etc, execute them, then exit. What you don’t want to happen is output anything. Just make sure you don’t echo any content to stdout (echo, loading views, etc.) If you are sending emails, you would load a view file, but return the contents to a variable instead of displaying it (pass true as third param), then run through normal procedures for sending emails.
One thing to be aware of regarding security, you probably don’t want anyone accessing /mailprocessor/sendmail from a web browser[In my case I don put the scripts on the web directory- so no need to worry about ], so in the mailprocessor.php controller constructor, you may want to do some kind of check to make sure we are coming from a cron script. Probably the easiest method would be to test the SCRIPT_FILENAME variable:

if($_SERVER['SCRIPT_FILENAME'] != 'mailsender.php')

  exit; 

8.Once everything looks good, add your crontab entry to execute the script, and there you have it! Sample cronjob [from cPanel  can easy do that- you just need to put the script location to run]:

# execute cronjob every day at 2am

00 02 * * * /HOME/myusername/misc/scripts/mailsender.php

My addition : 
8. You need to edit the BASEPATH on the mailsender.php.  You need to put the absolute path instead of relative path.


Original post - You will get some other alternative ways to do the things :)
Update-05/16/2011: On version 2, CodeIgniter has implemented a feature to run sccript from CLI. :) 

Some Popular Youtube Clone - Video Portal Scripts

For those of you who may be interested in setting up a niche video website based on the Youtube video sharing Social Media model - there are plenty of php scripts out there that will basically allow you to easily setup a Youtube clone website. I have worked with a lot of them - specifically the top popular ones, and this post is a brief review of video sharing scripts. These scripts can be installed on solo websites or within a sub-domain or folder within your current existing website. While there are now plenty of Video Share scripts out their (many of them just re-writes and refurbished clones of original ones), the scripts I review below are the ones that are on the top of the list.

PHPmotion -This is a free, opensource script and a good startig point if you do not have the money, but would like to experiment to see what kind of traffic your video niche portal can attract. I would not recommend it to anyone who is serious. It works well, but the design templates and the functionality is not that great. But, if you are on a shoestring budget and all you want to do is throw up something quickly and cheap - you can try this. Score: 7/10

Clip-Share - One of the initial players in this space. Has amazing graphical and layout templates that can be purchased in addition to the 4 basic templates that come pre-packaged. This is a solid script and a highly recommended choice for the serious. They sell 2 different versions - a Lite version at $99 and a Pro version at $199. You can purchase additional Web2.0 look-and-feel design template for $100 each. The template files can easily be customized as you wish. I had no problem whatsoever with their support and they were prompt to answer and get back to me with solutions to the problems I faced. If you have the budget go for this one. Recently GodTube.com that uses their script gota funding round of $30 million. Score: 9/10

Rayyz.net - This script is not just for video sharing. It is a complete Social Media website that has has video, photo, music, files, articles, games, blogs and community networking built into the dash board. If you’re looking for a more than just video and a full fledged script of Social Networking around various media - then take a look at this script. Athough, I have not personally tested this script or worked with it - it seems to be alright. the price is a bit steep at $418. If you are serious about setting up a Social Netowrking site around media sharing content, it would be better if you installed Boonex (which has a free version). However, dont strike out Rayzz from your list - as it is quite powerful in its own way (it focuses more on the Social Media sharing tools and aspect than the Social Networking aspect, which is what Boonex does). Score 6.5/10

Attachmax - This script will allow users to download the videos in addition to viewing them streamed. The script comes with a basic database of videos so you can have pre-populated content on your site. While the layout and graphic design may suit some people, I thought the script was pretty average and not too focused. At $$ , this may be a good buy to some of you. Score: 6/10

Alstrasoft Video Share Enterprise - Stay far far away from this one. Its got holes in it and the people behind it have a very bad reputation. I don’t want to bash them too much, but just do a search on Google with their name and you will see what other disgruntled people are saying on various forums. At $300 this is a complete waste of money (even though I would not pay even $10 for it). You have been warned! Score: 0/10

MediaMax Script - This script is a recent re-branding of Video Watch Pro. They have videos, pitcure, audio and social networking too - built into this script. The site and demo looks pretty impressive at $299 and they even have a Google Adsense flash video integration - which will allow you to earn more money within the flahs player. I have read elsewhere that their customer service is not up to mark - but this is not from a confirmed source. I think - you are good to go if you like their look and feel and functionality. Quite impressive. Score: 8/10

VShare - This is the script with the highest value-for-money at $10 !! The features are very advanced and their are plenty of them. I was quite taken by surprise to see the amount of features this script had packed into it - such as, alternate video server uploading, youtube direct embedding etc. The only drawback was the graphical template… it looked a bit ordinary. So, if you have the time and resources to alter template files - then i recommend this script highly. Although there are a few vendors on their site forums who are selling laternate templates, I am yet to see one which is “professional looking”. Also, I’d like to point out that their support and forums are extremely good and I got answers very quickly. Install this script if you have the ability to dabble around with the template and improve its basic look. Score: 9/10

osTube - This script is a media sharing model and offers Video, Photo, Audio, Documents, Community and Blogs. The pre-packaged templates look quite impressive and the functionality and video player look good. I don’t have any idea what it costs - but you should test drive their free version, which has limited features, before you buy their paid versions. They are definitely the most impressive “media sharing” scripts out their from what I can see. May not have as many functions as Rayzz (above), but definitely a better and sleeker dashboard - although I dont know how the engine is under the hood. On what i’ve seen and assuming this script works well, I give them - Score: 8/10

PHPmelody - If you’re looking to setup a Music Videos website, then this is script is fantastic. It does not allow you and your visitors to directly upload videos to the site - but one can embed videos from Youtube - which is a fantastic feature, as not only does it save bandwidth and server resources but it also allows you to quickly add content. They also have a neat “keyword” based search that the admin can use to find videos on Youtube and add them with one click! The script comes pre-packaged with thousands of pre-populated music videos - if you want to get content included right away. A breeze to install, and definitely worth the price at $39 with additional layout templates at $10 to $19. Support is excellent and prompt. Score: 10/10 if you are creating a Music Video website.

VidiScript - This one is a free script and it looked pretty decent (front end) - with ability to show pre-roll ads, insert text banner ads inside video player, a decent flash player, and two not-so-good light and dark templates (that can always be improved by you). VidiScript not only supports Videos, but now also supports Audio, Photo and Flash Game uploads. I dont know how stable this script is and how it will hold up for a busy site, but… hey… its free! and so definitely worth a try. Score: 8/10 (based on what I saw in the front end demo)

Clip-Bucket too. Its free and open sourced. I am not sure how secure it is and how it performs when measured up with the scripts above, as I had a quick look at the front end, but did not install it or use it on a site of my own. I must say for free price, the templates, the player the look and feel are HIGHLY professional and the folks behind it have done a great job in making a very neat user interface and front end. By the looks of it - this is going to be a killer video sharing script … and will probably put behind vhsare and phpmotion - if the backend lives up to the quality of the front end.

Mass video populator: It claims that it will allow you to mass populate and add videos to your video site (all scripts above are supported), in a few clicks. The price, I believe is the only downside as they have gone in for a bulk pricing, which works out to around 10 cents a video - steep I thought, for someone who wants to add say 1000 videos. Its a web based remote service.

VideoWatchPro: It is not that great, and scores around 5/10, from a brief look. I havent seen it in depth - but it doesn’t look to be anything worth spending more time on.

Conclusion: I suggest that you pick and choose from the scripts above and if you are selecting one not listed above you do your homework well - as their are many clones and copies of pre-existing scripts out their with holes in them. Start doing your research on this page of hotscripts.com - that lists a whole bunch of media sharing php scripts, and also make sure you visit forums such as Programming Talk to read reviews posted by other customers.

Invoking webservice using php

Yah, I just finished primary work for an online application. Basically I just learned how to use SOAP to access a webservice using php.
PHP 5 has introduced SoapClient class to do such things. Following is the code that I used :

<?php
$wsdl_url = "http://urltowebservice.wsdl";
$client = new SoapClient($wsdl_url);
$params = array('pasrm1'=>7, 'param2'=>'EN');
try {
$objectResult = $client->SomeMethod($params);
echo $objectResult->ResponseResult;
} catch (SoapFault $exception) {
echo $exception;
}
//var_dump($client->__getFunctions());


?>

Reference :
If you are interested about working with simpler SOAPS without using a wsdl file you can check the following:

• http://www.ibm.com/developerworks/opensource/tutorials/os-php-webservice/section3.html
• http://www.sks.com.np/article/8/xml-web-service-using-php-and-soap.html

Php page/URL redirect with header() function

If you are a php developer then you must know how to redirect a page from current page using php header() function. And example script should look like as follows :

<?php
header("Location: http://www.myurl.abc");
echo "I don know what to do next";
?>

The only thing that you should take care of is that NOTHING should be printed before the header() function call.
Now if you need to call this header function in the middle of any of your html design- what to do ? Simple call ob_start() at the very first line of your page then call have all your html tags,call your header() function and and call ob_end_flush() immidiately after header() function call. The code should look like as follows :

<?php
ob_start();
//Your html and or other code to print html
echo "A line before header() call";
header("Location: http://www.url");
ob_end_flush();
?>

Reference:

• header()
  
• ob_start()
• ob_end_flush()

How To : SEO friendly URLs

When you are working with web portals SEO is almost a must now a days. And when you are working for SEO, seo friendly URL is what you should take care of. Again when you are playing with php and Apache following are the ways that you can utilize :

• Using Apache mod_rewrite:

There is some debate as to whether or not this is actually a front controller design, since no index.php file is utilized. However, I thought it would be worthwhile to demonstrate how URL rewrites work at the highest level before moving onto advanced solutions. Remember, this is not the preferred approach, but it is a good primer. The following is an example of one-to-one relationship URL rewrites in an .htaccess. If you have less than a dozen pages on a small Web site, this can be a useful quick fix.

<ifmodule mod_rewrite.c>
RewriteEngine On
RewriteRule ^our-services/strategic-consulting$ /our_services.php?id=1 [L]
RewriteRule ^our-services/web-design$ /our_services.php?id=2 [L]
RewriteRule ^our-services/web-development$ /our_services.php?id=3 [L]
</ifmodule>

The IfModule conditional directive wraps a section of rewrite rules, and ensures that they will only get executed on a server that has mod_rewrite capabilities. If the module did not exist, and there was no conditional directive, users will receive an HTTP Error 500 - Internal Server Error. It is good practice to always use this directive.

According to the documentation, “The RewriteEngine directive enables or disables the runtime rewriting engine.” RewriteEngine can be declared multiple times within .htaccess, which can actually help with debugging. The directive can be turned on or off, and instead of commenting out or deleting a line during testing, it is best to use RewriteEngine Off. In the example there is only the need to turn the rewrite engine on once.

The RewriteRule directive is one of the most powerful aspects of mod_rewrite. I invite you to bookmark and eventually read the official documentation for RewriteRule. There are too many facets to this directive to cover them all, so I will mention just a few.

1. RewriteRule uses pattern substitution through regular expressions. So, the more you know about regular expressions, the more powerful your rewrite rules can become.
2. Each rule begins with RewriteRule, and multiple rules can be applied to a single URL string before the page request is passed along to a PHP (or other) file.
3. Definition order is important. Top down is the order of execution. You cannot skip rules unless they are commented out.
4. The first portion of a rule is the pattern to match. The caret anchor (^) designates the beginning of a pattern match (start of line). The dollar anchor ($) designates the end of a pattern match (end of line). (This is the URL that the user typed into the address bar.)
5. The second portion of the rule designates the file path that should actually handle the request behind the scenes if, and only if, a pattern match was found in the first portion of the rule. You can use regular expression groups and backreferences to pass information from the first portion of the URL to the second portion of the URL. More on that later.
6. The final portion of the rule is a flag. It provides additional flexibility without the use of further pattern substitutions. The flag in the example above is “last rule”, and states that if a match was found, no further rules should be processed. The request is then forwarded on to the application. Multiple flags can be separated by a comma, all of them contained between a single set of brackets.

• Writing own PHP engine for URL translation:

Step 1.
In your .htaccess file (If you don't have one, then create a new one in the folder where your index.php resides) add the following:

<FilesMatch "index">
ForceType application/x-httpd-php
</FilesMatch>

This simply tells your apache server to treat "index" in the url as "index.php". If you are using a server other than apache, then you will need to configure it to do this.

Step 2.
Implement the class below so that an instance is create each time index.php is executed. What this does is it takes the special url www.mysite.com/index/menuID_28-type_article and
splits it into www.mysite.com/index/ and menuID_28-type_article strings, then it creates an array:
array['menuID'] = 28
array['type'] = article
You cannot use $_GET['paramName'], instead call getItem($key) method in UrlFilter passing the name of the parameter you want, for exaple: $this->urlFilter -> getItem("menuID").
You can use any number of parameters, and name them as you wish, all you have to do is use the getItem() method and you got yourself nice URLs. Ofcourse you will need to build your links correctly for this to work, use getOriginalUrl() for this, then add your parameters to it: /param_value-param2_value2-param3_value3 and so on...

// constructs search friendly urls and create correct url paths for links and images
class UrlFilter {
var $origUrl;
var $numOfItems;
var $itemsMap;

function UrlFilter()
{
$this->processURI();
$this->constructBaseUrl();
}

function processURI()
{
$array = explode("index", $_SERVER['PHP_SELF']);
$newUrl = str_replace("/", "", $array[count($array)-1]);
$array = explode("-", $newUrl);
$num = count($array);
$this->numOfItems = $num;
$url_array = array();

$items = explode("-", $newUrl);
$num = count($items);
for ($i = 0 ; $i < $num ; $i++) { $item = explode("_", $items[$i]); $this->itemsMap[$item[0]] = $item[1] ;
}
}

function constructBaseUrl()
{
// need to construct the original url
$array = explode("index", $_SERVER['PHP_SELF']);
$num = count($array);
$this->origUrl = "";
for ($i = 0; $i < $num-1 ; $i++) { $this->origUrl = $this->origUrl . $array[$i];
}
}

function getItem($key)
{
if (array_key_exists($key, $this->itemsMap)) {
return $this->itemsMap[$key];
} else {
return null;
}
}

function getOriginalUrl()
{
return $this->origUrl . "index";
}

function getBaseUrl()
{
return $this->origUrl;
}

}

Curly use of cURL in php

I's interested in cURL in php since when Rashim was looking for something to get the yahoo weather feed as it is. He made it using cURL and I just loved it- amazed with versatile use of cURL.

I's looking for a way to get the all the statistics and searching facilities for expired or deleted domain names. And finally I decided to use cURL to search from some free service providers on the net although the usual approach I got is to store the names performing some scheduled search from who-is servers.
Anyway, here I'm gonna share all I've been playing with cURL :

Getting required html content from a website using cURL :

$url = "http://www.example.com/domains/detailedStatistics";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
$regex = '~(.*?)~s';
preg_match($regex,$curl_scraped_page,$match);
//var_dump($match);
echo "Detailed domain statistics:
";
echo "
". $match[0]."
";

?>

Searching on a website using cURL and parsing the html to get desired result:

I used a html form to post input values to a searcher file. The code on the searcher file looked like :

//A Beautiful html dom parser
include('./simple_html_dom.php');
$cat = explode("?",$_SERVER['REQUEST_URI']);

$url = "http://www.example.com/Domain_Name_Search_Results.aspx?".$cat[1];

$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);

// find all tabele tags with id=results
$html = str_get_html($curl_scraped_page);


$thetable = $html->find('table[class=BlackLeftRight]',0);
foreach($thetable->find('img') as $img)
{
$img->outertext = "";
}
echo "".$thetable->innertext."" ;

?>

That's it :) Happy curling then :)

Reference:

• PHP cURL manual
  
• Simple dom parser