Archive

Archive for the ‘PHP’ Category

Installing Memcache Server for PHP on Windows

February 15th, 2012

Memcached is a free open source, high-performance, distributed memory object caching system. It is currently used by a lot of websites, including Flickr, Twitter, Youtube, Digg and WordPress.

I’ve been using memcached on a few production servers, but never thought it could come in handy on a Windows development machine – in fact I didn’t even thought it was available on Windows. So, a few days ago, while working on yet another Magento project, I ran a Google search for “memcache windows” and it turned out there are a few Win32 ports of the original version. Cool! It can be used on Windows, now all I need is to find the right PHP extension.

Here’s a step-by-step tutorial on how to get memcached running in PHP on a Windows box. There’s are a bunch of really good tutorials out there, but I think another could only be helpful.

1. Go to http://splinedancer.com/memcached-win32/ and download memcached. I’ve used the 09.03.2008 binaries, memcached 1.2.4.
You can also use v1.2.6 that you can download from http://code.jellycan.com/memcached/ – I’ve used this one to update from the previous 1.2.4 and I can confirm that it works.

2. Unzip the downloaded file to any folder (i.e. C:/memcached). I’ve saved in the same place where I have Apache, MySQL and PHP.

3a. If you’re on Windows Vista, navigate to your memcached folder, right click on memcached.exe and click Properties, then click the Compatibility tab. In here, check the “Run this program as an administrator” checkbox.

3. Install the service by running memcached.exe -d install from the command line. (you should be in the folder where you unzipped the file(s), ie: c:/memcached)

By default, memcached has 64M available for caching and runs on port 11211. If you need to change any of these you can do it by editing the Registry – open regedit then search for HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Services/memcached Server. Change the ImagePath entry from "C:\memcached\memcached.exe" -d runservice to "C:\memcached\memcached.exe" -d runservice -m 256 -p 11222". This gives you 256 megs of memory and change the port to 11222 (on a development machine you probably don’t need to change the port, but who knows). See the full list of memcached options – not sure which ones apply to the Windows version too.

4. Now, that the service is installed and configured, let’s start it: memcached.exe -d start
To test that memcached is running you can try telnet’ing – telnet localhost 11211 (use the port you configured the service to listen to). If you can connect on that port, everything is fine.

5. Memcached is installed, but we need to use in PHP, so we need an extension. Here’s a list of sites where you can download the dll from:
http://downloads.php.net/pierre/
http://www.pureformsolutions.com/pureform.wordpress.com/2008/06/17/php_memcache.dll
http://kromann.info/download.php?strFolder=php5_1-Release_TS&strIndex=PHP5_1
shikii.net/blog/downloads/php_memcache-cvs-20090703-5.3-VC6-x86.zip

I had to install a VC6 dll, so I’ve downloaded the last one. Unzip it and put it in your php extensions folder. If you don’t know where this folder is, try this command: php -i | find "extension_dir". Add the dll in there, then open php.ini (to find it, you can run php -i | find "Loaded Config") and at the end add something like this:

[PHP_MEMCACHE]
extension=php_memcache.dll

Now restart Apache.

6. To test that memcached can now be used in PHP, use the following snippet:

$memcache = new Memcache;
if( !$memcache->connect('localhost', 11211)) { //change 11211 with the port your memcached is configured to listen to
  die('Could not connect!');
}

echo '
';
print_r($memcache->getStats());

If you see an array with a bunch of data in it, you’ve done it. If not, something went wrong at some point. Make sure memcached is running, then make sure you’ve got the correct php settings and extension.

As I mentioned at the beginning of the post, installing memcached was something that just hit me while working on a Magento project – for those familiar with Magento, you know how slow it can be sometimes, especially when you need to constantly refresh the pages to check html/js/css changes. So, I thought memcached would help speed up the parts on which I wasn’t working on – everything except layouts and html block cache. And so it did, development seems MUCH easier now – but all about this in another post, soon to come.

Many thanks for helping me figure this out and (hopefully) speeding up development to:
http://splinedancer.com/memcached-win32/
http://pureform.wordpress.com/2008/01/10/installing-memcache-on-windows-for-php/
http://shikii.net/blog/installing-memcached-for-php-5-3-on-windows-7/

PHP , ,

Patching the Magento USPS module

January 20th, 2011

Since their last update on Jan 3rd, 2011 USPS has stopped working on all of my Magento distro’s. The reason for that is a change in USPS’ response XML values. Magento has its part of blame here, first for not patching it immediately (which again shows how much they care about the CE users) and second for not coding it properly in the first place – they’re using the method name (i.e. Express Mail) to see if that method is allowed from the admin, instead of using the CLASSID attribute.

This issue has been reported on magentocommerce.com bug tracking system and it’s also discussed in the forum. USPS makes it clear that they’ll discontinue all API versions prior RateV4 ad IntlRateV2:
“All Rate Calculator API integrators are encouraged to migrate to the latest API versions (RateV4, IntlRateV2):
RateV4 and IntlRateV2 will be the only Rate Calculator API versions to offer the full range of new products and functionality
Rate, RateV2, RateV3 and IntlRate will be retired in May 2011, requiring all integrators to migrate to the latest versions”

As explained on the forums, the fix is quite simple, as USPS only added the <sup>®</sup> which is the encoded (twice 🙂 ) version of the registered sign in a sup tag. Magento values are defined without those so when the code tries to match them it fails. All you have to do is clean that string and you’re good to go. I don’t like to modify core files so the fix involves copying app/code/core/Mage/Usa/Model/Shipping/Carrier/Usps.php to app/code/local/Mage/Usa/Model/Shipping/Carrier/ and working on the local version of Usps.php. Strip the special chars:

$mailService = str_replace('®', '', strip_tags(htmlspecialchars_decode(htmlspecialchars_decode((string)$postage->MailService))));

and

$svcDescription = str_replace('®', '', strip_tags(htmlspecialchars_decode(htmlspecialchars_decode((string)$service->SvcDescription))));

 

Here’s the archive with the patch. Just download it and unzip it in the root of your Magento distribution. USPS patch for Magento 1.4.2.0 (1014 downloads)

 

Note: This patch applies to Magento CE 1.4.2.0. For all other versions you will have to do it manually as explained above.

 

Magento, PHP , ,

Reducing the number of CSS requests

September 11th, 2009

One common problem with larger/trafficked websites is that they end up serving a considerable amount of css data. Most of the frontend developers use individual style sheets for resets, structure, design, homepage / innerpages and so on. In fact, there’s no other rule here than keeping the css code as maintainable as possible. This also means adding comments and maybe writing every declaration on a single line, using indents.

Another well-known problem is with caching. When the CSS file is changed, most of the times you’ll need to clear the browser cache in order to display the site correctly, with the new changes applied. Well, for users that’s not an option and all the css upgrades should be reflected instantly. I know what you’re saying: there’s Apache’s mod_expire for that, but there are times when you cannot use that or don’t know about it or you don’t know how to get it running. If you fall into this category, this article should solve that.

Let’s take an example first. On the homepage of a site, there are 3 style sheet files requests. Their total size is 46,274 bytes – that’s roughly 46KB. My aim is to only request a single CSS file and make it change its name when anything in those 3 files changes. The site uses templates, so this tutorial will be based on that architecture – for plain ole’ php-html sites, this should be even simpler.

So let’s write a list of what needs to be done:

  1. Reflect any css change into the file name so browser’s cache is instantly invalidated
  2. Combine all requested files into a single file
  3. Create a queue of css files: both templates and controller files should be able to read & write to the queue.


Step 1 – Changes to the Template Model

In the template model, add new public methods for addCss( $cssFile, $media = 'screen') and getCompiledCss($media = 'screen'). Here’s how it could look like:

function addCssFile( $file, $media = 'screen' ) { 
	$this->cssFiles[] = array( 'url' => $file, 'media' => $media ); 
	return $this; 
}

function getCompiledCss( $media = 'screen' ) {
	if( $this->compiledCss instanceof CompiledCss ) { return $this->compiledCss->getFilename(); }
	if( !count($this->cssFiles) ) { return null; }

	$cssMedia = array();
	foreach( $this->cssFiles as $css ) {
		if( !isset($cssMedia[ $css['media'] ]) ) {
			$cssMedia[ $css['media'] ] = array();
		}
		$cssMedia[ $css['media'] ][] = $css['url']; 
	}
	if( !$media || !isset($cssMedia[$media]) || !count($cssMedia[$media]) ) { return null; }
	
	$this->compiledCss = new CompiledCss( $cssMedia[$media] );
	return $this->compiledCss->getFilename();
}

Maybe the getCompiledCss seems complicated, but in fact it’s not. It’s just a proxy method with some lazy-initialization in it. It only checks to see if the CompiledCss object is instantiated (creates it if it’s not), then return the compiled CSS file name. So basically, everything is done in the CompiledCss class.


Step 2 – Changes to the template

All valid and W3C-compliant CSS declarations go in the header of the HTML, so it should be easy to group all CSS requests together and call the getCompiledCss somewhere in the main layout file or in a head template component. Either way, here’s a quick peek at how the templates could look like:

<?php 
	$this->addCssFile('general.css', 'screen')
		->addCssFile('homepage.css', 'screen')
		->addCssFile('panels.css', 'screen');
?>
<?php if( $this->getCompiledCss('screen') ): ?>
	<link type="text/css" rel="stylesheet" href="<?php echo $this->getCompiledCss('screen'); ?>" media="Screen" />
<?php endif; ?>


Step 3 – CompiledCss Class – main logic

As we’ve seen above, 99% of the logic is done inside this class, so tasks #1 & #2 are its responsibilities. First of all, remember that inside the Template Model there was only a call for getFilename() – let’s take a look at it:

function getFilename() {
	if( null === $this->filename ) {
		$this->compile();
	}
	return $this->filename;
}

Again, it’s pretty simple – all it does is to call the compile() method once. Otherwise, if this method is called more than once, it just returns the same file name. As you can see, complie() does the entire job, actually, so on with it:

First, make sure each css file gets compiled only once, then sort their names alphabetically, so that the generated file name is always the same, no matter the order you add the files (I know this can cause some serious issues – but bear with me until the end for problems, solutions & improvements

$this->cssFiles = array_unique($this->cssFiles);
asort($this->cssFiles);

The file name will need to parts: one for recognizing the css files that are “inside” and another one for content checksum – the second part helps generating new file names when the source files are modified.

$filename = $out = '';
foreach( $this->cssFiles as $css ) {
	$filename .= Crc32($css); 
	$cssPath = 'path/to/css/files/' . $css; #whatever path you need to get to the css files
	if( is_file($cssPath) && is_readable($cssPath) ) {
		$out .= file_get_contents($cssPath);
	}
}
$filename = Crc32($filename);
$checksum = Crc32($out);

We need to cache this compiled css, to be able to serve CSS data. As opposed to the method of requesting a list of css files on a GET request, this one needs to save the parsed css someplace, otherwise decoding the checksums would be almost impossible. So, write the “processed” css data to a file and than load it each time it is requested published here. Here’s the code for actually compressing the css:

$output = preg_replace('#[ \t]+#', ' ', $output); #replace multi-spaces or multi-tabs with a single space
$output = preg_replace('#[\n\r]+#', '', $output); #remove new lines
$output = preg_replace('#([:;])\s+#', '$1', $output); #remove spaces after : and ;
$output = preg_replace('#/\*(.*?)\*/#', '', $output); #remove all comments
return $output;


Step 4 – Final touches

One more step is required for this to work. You need a php file handling the compiled css request. Let’s assume your requests (and compiled file name) look like this: crc32-crc32.css. That means you’re gonna create a mod_rewrite rule in a .htaccess file mapping 8 chars, dash, 8 chars, dot css to a php file, passing both checksums as parameters. Inside the php file, just send some headers and request the file:

header('Content-Type: text/css');
header('Cache-Control: cache');
header('Pragma: cache');
header('Expires: ' . gmdate('r', strtotime('+1 year')));
echo CompiledCss::request( $firstChecksum, $secondChecksum );  

request() logic is quite simple – check for existing cache and display it, otherwise just return null. Usually, if the process runs normally, first we’re gonna get the getFilename() call which saves the cache and returns a valid file name, and just after that the request() method fires.


Compression performance

Like I said in the beginning of this post, the original file size for all three requests for this example was roughly 46K. After compilation it got to 42K – that’s around 91% of the original size, aprox. 10% compression.
Best compression level with this tools was 13%, but the usual rate is 10%. Keep in mind, however, that it depends on how the css file is written. For example, I write each declaration inline, so there are only a few spaces and new lines, but for indented css the compression could reach 20% (tested).

So there you have it. At least 10% improvement, a few extra requests saved, and the ability to invalidate browser cache on-demand. I’ve been using this tool for a while now and I can tell you I have no design/layout problems or conflicts and the total size of css requests have saved enough bandwidth – not to mention that download times are reduced. Add mod_deflate or manual gzip compression to it and it’s gonna download even faster.


Problems, Solutions & Improvements

I know some of you might have seen only problems in this article. Or you might thing this is way too complicated and it’s not needed cause there are other ways (much more simpler and faster) of doing it. So let’s see what problems I’ve identified so far and what you can do:

Problem: if CSS files are sorted and they are not compiled in the same order as they’re given, there might be conflicts, overlapping styles, and in the end the design might be screwed up.
» Solution(s): One of them is to improve your css declarations making them work the same, no matter how they’re parsed. The other solution is to remove the sorting in the compile() method, but be sure to always use the same queue push to get the same filename. For example, pushing a.css, b.css in one place an b.css, a.css in another place will create two separate css cache files.

Problem: crc32 only uses 32 bits, so checksum collisions might occur (although the chances are minimal)
» Solution: Switch to md5 or even sha1 if you think those don’t create collisions.

Problem: when stripping spaces after :;, there are chances to affect strings, like for instance url’s.
» Solution: try to keep your filenames free of : and ;, or, if this isn’t enough, remove this line: $output = preg_replace('#([:;])\s+#', '$1', $output);
The same applies for removing multi-spaces and multi-tabs.

Problem: removing comments might break some IE hacks
» Solution: hacks are not recommended anyway, so try using separate IE6- and IE7+ stylesheets, if nothing else works. Again, removing that line from the output parsing helps, but eventually the compression level would be zero.

Improvement: gzip the css and server gzip content if the client accepts gzip. Try enabling mod_deflate anyway, if load is not really an issue.

Improvement: check you inodes cache settings to help speeding disk reads – as you’ve noticed we’re reading all css files each time a page is requested, so some system tunin might help.

PHP , ,