Nicholas Skinner

Finally got around to writing up / making available for download a number of projects that I have for sale. Some are new such as Takeaways Direct, and some I have had around for a while such as Parallel Track but have not made the source code available before:

• Takeaways Direct – An online restaurant and takeaway ordering application coded in PHP.
• Parallel Track – A vehicle tracking application coded in PHP, available (currently) at no cost for personal non commercial use (commercial use requires a license).
• Telit Module Remote Update Script – A Python application that can be integrated into your own Python applications on the Telit Modules to enable remote software updates via HTTP and GPRS.
• Animal Tracking Script – A Python application for the Telit modules suitable for animal GPS based tracking. It can however also be used in any situation where low power consumption / batter operation is required.

Setting up Apache securely for multiple system users each with their own domain name(s), and subdomains on CentOS 5.

This post is a follow on to: Setting Up Exim Mail Server For Multiple Domains, it describes the configuration I am currently running on my own server for web hosting.

I looked into various options, and in terms of balancing features (i.e. not restricting the functionality that can be used in PHP), security (not using e.g. PHP Safe Mode that is no longer recommended, or requiring 777 permissions on directories that need to be writable), install complexity, and maintainability for CentOS 5 this seemed to be the best approach. All software is sourced from standard CentOS repositories therefore can be installed / updated easily via “yum”. The trade-off of this approach is performance, PHP is running as CGI therefore will not be as fast as using PHP as a module or using something like FastCGI.

To keep things secure each person who is in control of a particular domain name/names will be setup with a single user account on the system. The hosting space for these domains will then be placed in that users home directory. This approach has the advantage that there is not a direct link between domains and user accounts, meaning that a (perhaps more traditional) one domain per user account approach can be taken but also has the advantage that if a single person has multiple domains they can all be setup in a single user account meaning there is no need to e.g. setup multiple SSH/SFTP/FTP logins on the client side, and on the server side those domains can access each others files/resources without any permissions issues, or alternate workaround being needed. Each user account has a single Apache configuration file meaning a single place to e.g. setup a WordPress MU install for a specific user on the system.

Install Packages

1. Install Apache:

   yum install httpd

2. Install PHP, and any additional PHP extensions you need:

   yum install php
yum install php-gd
yum install php-mysql
yum install php-mbstring
yum install php-imap
yum install php-soap
yum install php-xml

Configure Apache

1. Set the daemon to start on bootup, and start the service:

   chkconfig httpd on
service httpd start

2. Create directory to hold virtual host configuration files:

   mkdir /etc/httpd/virtualdomains
chmod 700 /etc/httpd/virtualdomains

3. Create directory to hold virtual host log files:

   mkdir /var/log/domlogs
chmod 701 /var/log/domlogs

4. Add the following in at the end of your “/etc/httpd/conf/httpd.conf” file:

   #Turn off default following of symlinks (will turn it back on later as SymLinksIfOwnerMatch)
<directory />
Options -FollowSymLinks
</directory>

   #Provide default install page
   <LocationMatch "^/+$">
   Options -Indexes
   ErrorDocument 403 /error/noindex.html
   </LocationMatch>

   #Allow system users to make use of various options such as SymLinksIfOwnerMatch (for mod_rewrite etc) in their .htaccess files
   <Directory /home>
   Options +Indexes +SymLinksIfOwnerMatch
   AllowOverride FileInfo AuthConfig Limit Indexes
   </Directory>

   #Enable PHP (only for .php extension), and add it to pages considered as index
   <FilesMatch \.php$>
   SetHandler application/x-httpd-php5
   </FilesMatch>
   AddType text/html .php
   DirectoryIndex index.php

   #Virtual hostnames
   NameVirtualHost *:80

   #Default page for domains not setup
   <VirtualHost *:80>
   DocumentRoot /var/www/html
   </VirtualHost>

   #Include all the individual virtual host configuration files for processing
Include virtualdomains/*.conf

5. Comment out everything in “/etc/httpd/conf.d” this default file runs PHP as a module.

6. Restart Apache such that it reads in the new configuration:

   service httpd restart

User Account Setup

Follow this setup process for each system user account that will be hosting a website / websites.

Note: The system user account “nstech” is used in this example, it is assumed this system user has already been setup using e.g. “useradd nstech”.

1. Create a file (will be blank initially) to hold Apache virtual host configuration:

   touch /etc/httpd/virtualdomains/nstech.conf
chmod 644 /etc/httpd/virtualdomains/nstech.conf

2. Allow all system user accounts (inclusing the “apache” user account) to recurse the directory tree:

   chmod 711 /home/nstech

3. For security reasons suEXEC has a number of restrictions on what it will allow to be executed, therefore each user will need a copy of the PHP binary that has its “user” and “group” set to that accounts username, has appropriate permissions, and is placed in a suitable directory such as “/var/www/cgi-bin” (suEXEC has a hard coded path that it is not possible to change without recompiling). However copying PHP for each user is inefficient, therefore e.g. a bash script can be used that then calls the PHP binary itself:

   mkdir /var/www/cgi-bin/nstech
chown nstech:nstech /var/www/cgi-bin/nstech
chmod 755 /var/www/cgi-bin/nstech

   /var/www/cgi-bin/nstech/php-cgi.bash
   ------------------------------------
   #!/bin/bash

   /usr/bin/php-cgi "$@"

   (script from Stuart Herbert's PHP Blog)

   chown nstech:nstech /var/www/cgi-bin/nstech/php-cgi.bash
chmod 755 /var/www/cgi-bin/nstech/php-cgi.bash

4. Create a directory to hold all log files for the users domains:

   mkdir /var/log/domlogs/nstech
chmod 750 /var/log/domlogs/nstech
chown root:nstech /var/log/domlogs/nstech

Domain / Subdomain Setup

Follow this process for each new domain or subdomain that will be setup in a users account.

Note: The domain name “www.ns-tech.co.uk” and user account “nstech” are used in this example

1. Create a directory on the file system that will serve as the webspace for the domain being setup, and set appropriate permissions:

   mkdir /home/nstech/www.ns-tech.co.uk
chmod 750 /home/nstech/www.ns-tech.co.uk
chown nstech:apache /home/nstech/www.ns-tech.co.uk

2. Create (initially blank) log files for the user, and give the user permissions to view them (e.g. read them via PHP scripts or SSH).

   touch /var/log/domlogs/nstech/www.ns-tech.co.uk-error_log
chmod 750 /var/log/domlogs/nstech/www.ns-tech.co.uk-error_log
chown root:nstech /var/log/domlogs/nstech/www.ns-tech.co.uk-error_log

   touch /var/log/domlogs/nstech/www.ns-tech.co.uk-access_log
   chmod 750 /var/log/domlogs/nstech/www.ns-tech.co.uk-access_log
   chown root:nstech /var/log/domlogs/nstech/www.ns-tech.co.uk-access_log

   touch /var/log/domlogs/nstech/www.ns-tech.co.uk-bytes_log
chmod 750 /var/log/domlogs/nstech/www.ns-tech.co.uk-bytes_log
chown root:nstech /var/log/domlogs/nstech/www.ns-tech.co.uk-bytes_log

3. Append the virtual host configuration onto the existing virtual host configuration file for this user:

   /etc/httpd/virtualdomains/nstech.conf
-------------------------------------
<VirtualHost *:80>
SuexecUserGroup nstech nstech
DocumentRoot /home/nstech/www.ns-tech.co.uk
ServerName www.ns-tech.co.uk
ServerAlias ns-tech.co.uk
ErrorLog /var/log/domlogs/nstech/www.ns-tech.co.uk-error_log
CustomLog /var/log/domlogs/nstech/www.ns-tech.co.uk-access_log combined
CustomLog /var/log/domlogs/nstech/www.ns-tech.co.uk-bytes_log "%I %O"
Action application/x-httpd-php5 "/cgi-bin/nstech/php-cgi.bash"
</VirtualHost>

4. Restart Apache such that it reads in the new configuration:

   service httpd restart

Note: As far as I am aware the setup process described is reasonably secure however as with securing anything somewhat complex such as PHP, with Apache, and multiple system users but still allowing a reasonable amount of functionality there are a number of thing to consider. You are advised to test the security of this setup yourself in case I have missed something. Any issues please leave a comment below.

I was recently asked to put together a high spec’ computer primarily for gaming.

I usually use Intel motherboards, and since the recipient was keen on a processor using the LGA 1366 socket this meant the DX58SO X58 Express Chipset board was the only option.

I went with the following components:

I first started running my own dedicated server a number of years ago. At the time I was not particularly impressed with the control panel software available (Plesk, cPanel) as it seemed to somewhat take over the server and then meant that if you had any custom requirements there was always the extra task of checking if they were possible to do with the control panel / how to do them in a way that was compatible with the control panel. Also there was the extra cost of licensing the control panel software. I therefore decided to configure the server from scratch.

This post discusses setting up SMTP services using the Exim mail server software in a virtual hosting environment with support for:

• Multiple domains
• Multiple accounts (mailboxes)
• Catch all accounts
• Authenticated SMTP relaying
• Forwarding
• Forwarding to multiple addresses (lists)
• Forwarding of messages to system users to external accounts
• Bouncing email with/without a custom message
• Blackhole’ing email
• Using Postini spam filtering (blocking email sent directly to the server)

The server was setup such that there were several “customers” who each had their own user account on the system. Mail for accounts is stored in users home directories under “/mail”. Each customer can have one or more domains.

Background

Exim is very powerful however unfortunately that power comes at the cost of complexity in terms of setting things up. Its not just a case of point and click, or adding a line to a text file for new accounts, you first have to actually setup the rules in Exim that tell it (e.g. in the case of accounts) to accept messages from domains hosted on the server in the ACL, “route” that mail to a custom “transport”, configure the transport to (depending on recipients address and domain) lookup where to store the new mail on the server.

After a reasonable amount of time spent both reading the Exim documentation and searching on the Internet I was able to come up with a configuration supporting the above features centred around the following text files / directories:

• “/etc/exim/authrelay” – A list of usernames/passwords allowed to relay mail.
• “/etc/exim/userdomains” – A list associating domains back to user accounts on the system.
• “/etc/exim/domains” – A directory containing files for each domain name that themselves contain a list of mailbox accounts for that domain.
• “/etc/exim/virtual” – A directory containing files for each domain name that themselves contain a list of forwards (including catch all, bounce, blackhole) for that domain.
• “/etc/exim/postini_filtered” – A list of domains where mail should only be accepted for delivery from the Postini spam filtering service.
• “/etc/exim/userforward” – A list of local system user accounts for which mail should be forwarded to alternate accounts to cater for system services such as cron which by default send results of commands run to e.g. useracctname@localhost.

I recently had a requirement come up for a web application involving communicating a certain set of information over the telephone and receiving a response from the party on the other end acknowledging receipt of that information. I looked into various possibilities, the main one being Asterisk “the world’s leading open source PBX” (according to their website) however that would have involved a reasonably amount of setup in terms of the server, a VoIP provider, and actually getting TTS (Text to Speech) working using e.g. Festival.

I therefore started looking into alternatives, and came across the following:

• AQL
• Messaging People
• CDYNE Phone Notify

I chose CDYNE Phone Notify, it basically provides a VoiceXML type service however it does not use the VXML standard.

See below for some of the advantages, disadvantages, and other things to note that I found when testing it out and integrating it into a PHP based web application:

Advantages:

• Straight forward to interface with via SOAP (I used PHPs standard SOAP extension).
• Easy to code scripts ranging from the simple to the more advanced.
• Advanced script can be interactive and dynamic, taking user input via DTMF, sending it to a server you specify via HTTP and then proceeding with the script depending on the response received from the server.
• Range of voice languages, ages, genders available.
• Retries calls automatically up to 3 times.
• Debugging via email containing log of what happened via the call.
• Possible to change voices, voice speed, and insert pauses as and when required during a call.
• Handles inbound calls*
• Can transfer calls back to the PSTN depending on user input*
• Possible to download converted TTS MP3 file*
• Can record speech over the phone*

* Not features I had a need for / tested.

Disadvantages:

• Charge is made even if calls are not successful, i.e. recipient does not answer (a limited number of retries are however available at no extra charge).
• I have heard more realistic TTS sounding voices, and eventually went with a US voice because they seemed clearer / easier to understand than any of the UK voices available, rates are reasonable / inline with other VoIP providers for the UK.
• They operate a payment system whereby your credit card is billed depending on your usage at the end of each month. This opens up the possibility of an application e.g. getting stuck in a loop and this resulting in a very large bill at the end of the month. A pre pay system would avoid this. However apparently they do monitor accounts for out of the ordinary usage levels.

Other things to note:

• Surcharge for international calls, as it is a US based service (however this is somewhat to be expected).
• Wiki contains all the documentation you need however it could be organised in a way that makes it easier to fine.
• Documented commands “StatusChangePostUrl” “MaxCallLength” do not seem to work when specified on the “NotifyPhoneAdvanced” SOAP call, only when specified on the script as “~SetVar(maxcallseconds|60)~” and “~\StatusChangePostURL(example.com)~”.

Overall although it has some disadvantages I was impressed with the features available from the CDYNE Phone Notify service and the ease at which they can be utilised in PHP.

I have recently been testing out the iPhone applications “Trails“, and “Every Trail” for recording bike trips, along with geo-tagged photos.

View Trail Recorded With:

• Every Trail
• Trails

Every Trail Features:

• Uses Google Maps (requires an Internet connection to show your position on a map).
• Shows current position only on map (not a track log) and shows where photos have been taken.
• Allows you to take a photo while recording the trip (however requires unlocking / relocking the application each time).
• Features seamless integration with the Every Trail website, with uploading of geo-tagged photos directly from the phone.
• Shows basic trip information.

Trails Features:

• Uses maps from Open Street Map (does not require an Internet connection – maps can be pre downloaded for specified areas / zoom levels over WiFi before you leave).
• Shows current position overlaid on map along with a track log showing where you have been.
• Can change map type to “Terrain + Cycle” to show some cycle trails depending on your location.
• Has handy default settings to choose from depending on if you are “Jogging”, “Hiking”, “Biking” or “Driving”.
• Allows you to take a photo while recording the trip (without unlocking / relocking the application each time – this significantly streamlines the process of taking photos).
• Can upload trips directly to the Every Trail website however does not upload geo tagged photos (photos must be manually uploaded).
• Allows basic editing for tracks directly on the phone (along with removal of inaccurate GPS readings, and labelling of waypoints).
• Has the ability to import GPX files (however the file I tried to import did not seem to work properly).
• Shows comprehensive trip information including speed/altitude graphs when you turn the phone sideways.

The ability of Trails to download maps on e.g. a fast WiFi signal before starting is a major advantage, and made navigation much quicker. With Every Trail you could otherwise be left with either no map in the case that there is no GSM signal, or having to constantly wait around for maps to be downloaded over slow GPRS (if you are in a remote location with no 3G). This is likely due to its use of less license restrictive Open Street Map mapping data. Use of Open Street Map also means some cycle trails are also available in Trails, and if it happens to show the route you are taking, then navigating it with your iPhone becomes a whole degree of magnitude more straight forward.

A major disadvantage with Trails however is that it is not very geo tag friendly – photos do not have any EXIF taken date, or EXIF latitude / longitude information saved to them. It is worth noting that photos taken with Every Trail also seemed to be similarly lacking in either an EXIF taken date or latitude / longitude however Every Trail appear to have somehow got around this issue in a way that Trails has not because it does not have the ability to upload photos directly from the phone. Every Trails handling of photos on the phone itself however is less than ideal. It shows an icon on the map of where you took the photo but I found this unnecessary, it also seems to slow down response time when viewing / moving the map. If I had taken any more photos this would have likely become more of an issue.

I thought the geo tagging issue with Trails may have been due to not having something configured correctly, however after checking the developers site he has a tutorial on how to geo tag photos using software on your PC or Mac. I am therefore assuming the application just does not support geo tagging. I used the “Geosetter” application recommended on the site, and the only way I could get photos taken on the iPhone to show up correctly was to set the photo taken datetime to the file creation datetime manually for every photo (photos taken using a Sony DSC W5 did not have this problem), a somewhat tedious and time consuming process, and then photos still had to be uploaded manually to the Every Trail website.

Both applications resulted in the battery draining much faster than it otherwise would however this was to be expected. Admittedly I did not follow the recommended advice of turning off WiFi and 3G, and turning down the backlight (may not be very feasible on a sunny day anyway) however if you are going to use either application for longer than 4 hours and need to make calls and still need to use the phone normally afterwards then you may need to look into an external battery pack or plug in battery booster.

Overall if your main goal is showing a trip with geo tagged photos taken with the iPhone, you can put up with the slightly longer winded approach to taking photos (unlock/relock phone each time) and you are not really going to be relying on the phone for navigation, the Every Trail iPhone application is the simpler, more straight forward approach.

If however you are looking for a much more polished, and feature complete application that can show you in real time a range of statistics, you will be using the iPhone for navigation, and either not taking photos, or using a separate digital camera (which you will correctly set the time in before departing) Trails is the better application.

I recently went to the Isle of Wight on holiday, and while there went on two bike trails which I recorded using the "Trails" iPhone application (see below). I am planning on posting a review of Trails versus Every Trail shortly.

Isle Of Wight – Yarmouth to Freshwater (Route 1)

Widget powered by EveryTrail: GPS Community

Isle Of Wight – Sunshine Trail (Route 13)

Widget powered by EveryTrail: Share GPS Tracks

Geo-tagged photos also available on Flickr:

When running an AdWords campaign part of the process for improving ROI often involves adding in a number of broad match keywords, then setting up Google Analytics and using the Override Bid Term Filter to find out the exact keywords that triggered a broad match. The end goal being to remove the broad match keywords and add in exact/phrase match keywords in their place which are usually both more effective (they are more targeted) and therefore more cost effective.

With a new campaign using Google Analytics is the simple solution however I recently took on the job of assisting a client who had already been running a campaign, but did not have the “Override Bid Term Filter” installed. They did however have Apache log files, therefore I wrote a PHP script to extract the search terms from the referrer reducing the cost of running a further broad match campaign in order to determine exact/phrase match keywords to bid on.

[Download]

I have been growing increasing frustrated with Microsoft Outlook which I have been using as my primary email client since around 2000.

• Often crashes, especially if used with IMAP accounts.
• Never shuts down properly (.pst files are ~2.5GB).
• Searching is very poor / slow.
• Generally does not seem to be very fast.

Switching to another application was therefore not a decision I took lightly, especially since with a number of different accounts (I am involved in a number of different businesses), it is handy managing them all in one place. Another reason for switching however was that they are mostly all POP3 accounts, and I recently purchased an iPhone which works better with IMAP than POP3.

I considered Zimbra however although there are hosted solutions available for individual users it does not seem to be primarily aimed at that market. I therefore decided to go with Google Apps. Google Apps supports using your own domain name e.g. “@example.com” rather than “@gmail.com”, along with calendar, and contact syncing to the iPhone. To set it up you just need to signup and switch over your MX records to Googles servers / verify you own the domain with Google (either setup a CNAME record or put a text file on your web server).

I am still in the process of moving over accounts to GMail and have moved 4 so far. I was hoping to just have a single GMail account however in the end this has not worked out. GMail does allow you to add multiple addresses to “Send mail as” and has a “Reply from the same address to which the message was sent” feature however some email clients such as Outlook 2007 will show “From yourusername@youmainmailaccount.com on behalf of yourusername@yourdomain.com” which does not look particularly professional. The other issue is that the iPhone does not seem to allow setting up multiple senders.

I have therefore currently setup two Google Apps accounts, one for my main email and 1 for my other 3 businesses. They are all liked in some way therefore it is not too much of an issue for mail to show the “on behalf of” line.

Setting up multiple Google Apps accounts I was expecting that cookies may prove to be an issue i.e. I might have to keep signing in/out of different accounts (or use the Firefox Profiles workaround) as you would if you have multiple regular Google Accounts, however it turns out this is not the case as cookies are handled on a path level / you login at e.g. “http://mail.google.com/a/yourdomain.com”

Setting up multiple GMail accounts however then leads to another problem, GMail Notifier only supports a single account. I am therefore planning on setting these account up in Thunderbird and running it just to get the desktop notifications (i.e. I will see the notification then login to webmail to take advantage of conversation view / searching etc).

I have been using GMail for a few weeks now and on the whole am very happy with it however there are a few issues I have experienced regarding sending executable program files via email, and backing up email.

GMail does not support sending or receiving .exe files via email, even in .zip files therefore I have resorted to setting up a send only account in Outlook for sending messages although have not yet found a way around the receiving issue (luckily however I generally do not receive many .exe attachments via email).

In terms of backups I looked into a few options however am not sure there is a perfect solution available at the moment. A common solution seems to be running Thunderbird and using the Work Offline Synchronise feature but this did not seem too elegant to me. I therefore looked into a few software applications.

• MailStore – Looks like the best however does not support automatic backups (scheduling) in the free version (paid version is 349 USD).
• Forwarding messages to another POP3 account, or using e.g. Fetchmail to retrieve them via POP3 is a possibility however does not preserve labels or sent items.
• g-archiver – A viable option however after their recent issue with sending peoples passwords to one of their developers I thought it was best to avoid.
• GMail Backup – I chose to use this in the end. It can be used from the command line and therefore easily scheduled however does seem to store messages with long names (as separate files) in a single folder which is not particularly ideal as I now have a folder with +6,000 files (mail for around 1 year as when switching to GMail I imported mail from Outlook) that is not great from a performance point of view.

A PBX is basically a piece of equipment that allows you to get extra functionality from your telephones e.g. calling other extensions, forwarding calls to other extensions, conference calls, music on hold etc.

I have used the Linux based PBX software Asterisk since around 2003. Asterisk is different to most standard PBXs in that rather than being a closed platform allowing only a fixed set of options, Asterisk gives you the flexibility to program in software how calls should be made, received, answered, or routed either via the internet (for reduced call charges – particularly to international destinations) or via regular analogue phone lines.

The IP04 is dedicated hardware device suitable for running Asterisk that has 4 ports allowing for connection to e.g. standard analogue telephone lines, standard analogue telephones or the GSM network (via a GSM1 add in module), along with IP phones, and IP trunk providers over the network interface.

My reason for purchasing the IP04 was that I would prefer not to have to continue to maintain a full PC just to run a few phones, and also (not despite a lack of trying to solve the problem) I was having continuing intermittent issues with echo cancelation on calls made / received using analogue lines.

In terms of hardware it appears to be a solid, well built unit although I would question why the serial port was made an optional attachment (supplied on purchase) rather than just built in, and why no instructions are provided on how to open the unit to install additional modules (extra screws are hidden under the rubber feet).

In terms of software however setting it up turned out to be a whole lot less straight forward than I was expecting. I experienced the same issues getting it working (the main one being hangups not being correctly detected which required manual changes to zapata.conf) as I did back in 2003 with my original Asterisk install but this time with a number of extra issues caused by Asterisk being run on the IP04 and not a regular PC:

• MGCP protocol not supported by default – Admittedly most people will likely not use this any more however I was. Fortunately I had replaced 3 of the phones previously relying on MGCP with IP phones therefore only leaving one which I switched to using an IAXY I had spare.
• Does not seem particularly secure by default, e.g. telnet turned on with no password (you can install the login package “ipkg install login” to set a password).
• Web based configuration software installed on my unit by default did not allow setting of the unit IP address. You can however install other web based configuration software (“ipkg remove asterisk-gui” “ipkg install voiptel-gui” that does support this).
• If you do install the “voiptel-gui” web tool then you must use it to set the IP address, and not try to set it manually via editing “/etc/init.d/network-static”, since the voicetel-gui will overwrite the manual setting.
• The zapscan utility run in “/etc/init.d/zaptel” will rewrite your zapata.conf file, which if you are using more than one analogue port, and need to set a different “context” for each can be an issue. I solved it by adding to a script to undo the changes done by “zapscan” just after it runs, although this is obviously not the best solution. Simply commenting out zapscan did not work because that caused all analogue ports to stop responding.
• If you want to use Asterisks’s voicemail to email feature you need to make sure “ssmtp” is installed “ipkg install ssmtp” and configure it correctly in “/etc/ssmtp/ssmtp.conf” along with editing “/etc/asterisk/voicemail.conf” and adding in “mailcmd=/bin/ssmtp -t”
• If you would like to use music on hold you need to first convert it to a suitable format before copying it over to the unit.
• Documentation does not seem that comprehensive.

Overall however after resolving the issues above I am happy with the unit, as it has satisfied the two main objectives I had when purchasing it, i.e. no longer needing to run a full Linux PC, and much improved echo cancelling which I had not been able to resolve previously. If you are purchasing this product you will very likely need at least a minimal amount of Linux experience to get it setup and working correctly.