Archive

Author Archive

Disabling event observers programmatically in Magento

May 19th, 2015

I started using unit testing on a project and it took a while to figure out why my tests were not running. Every time I ran phpunit it only ran the first test and then nothing happened:

PHPUnit 4.6.6 by Sebastian Bergmann and contributors.

Configuration read from [...]

F

No matter what the assertion, the outcome was the same, the script was ending after the first test. Nothing in the logs, nothing in the system logs, total darkness. I installed the EcomDev_PHPUnit module several times (different forks and branches), same result. After intense debugging I found the problem to be with an event listening to controller_front_init_before, something that was checking the base url again and performing some redirects. So I had to somehow prevent that code from running in test mode.

Disabling events in magento is simple. Just add <type>disabled</type> to any observer and that’s it. But what if you need to do it at run time? What if under certain conditions certain events need to be unobserved? Turns out, this is not exactly easy – or at least not as easy as I’d expect, nothing like Mage::app()->unobserve('event', 'observer_name').

Events are being dispatched by the Mage_Core_Model_App class, but the list with all events is kept in Mage_Core_Model_Config. So, the easiest way is to just set (or add) a certain node on the configuration xml:

        Mage::app()->getConfig()->getEventConfig('global', 'controller_front_init_before');
        $path = 'global/events/controller_front_init_before/observers/fix_frontcontroller/type';
        Mage::app()->getConfig()->setNode($path, 'disabled');

That works just fine if you’re going through the regular flow and the config has been initialized (ie Mage::app()->getConfig()->init()). If you run this too early and the config has not been initialized you set this node for nothing, it will just get erased when the config xml is generated. When running phpunit with EcomDev_PHPUnit, things get a bit more complicated, since objects need to be mocked, espacially the app and config objects and config init is ran several times.

It turns out that listening to phpunit_suite_start_after does the trick. Put the code above in a listener and you should be ready to test.

Another approach involves the use of reflection class, but I wouldn’t use that unless there was no other way. Here’s the approach, just for the sake of it:

        $app = Mage::app();

        $class = new ReflectionClass(get_class($app));
        $property = $class->getProperty('_events');
        $property->setAccessible(true);
        $events = $property->getValue($app);

        $events['global']['controller_front_init_before']['observers']['fix_frontcontroller']['type'] = 'disabled';
        $property->setValue($app, $events);

Again, same assumption – config has been initialized.

And the last approach, and certainly much easier whenever possible: just hack the code and add a flag. Something along the lines if !registry(flag) then run this code;. This is a really simple solution, but it doesn’t work with code that cannot (should not) be changed (mainly because it would get overwritten on upgrades), hence the approach above.

Hope this helps, it took me good hours to figure everything out.

Inspired from the answers here

Magento

Proofpoint.com misinforms their audience, makes false malvertising allegations

September 12th, 2014

On Sunday, Sep 7 2014, 11:10am CentralPark.com received a message from Wayne Huang (whuang@proofpoint.com) from proofpoint.com informing us that we’ve been hit by malware. To be more precise he was saying our Ad Server had been compromised. Here’s the entire message we received:

I’m writing to inform you that your website has been hit by malware. Your OpenX ad server, used to serve out the ads on the upper-left corner of your website, has been infected.
Specifically, it is this URL: [url removed for security reasons]
Please react asap. If you need help you are welcome to reach out to me at whuang@proofpoint.com.

We take security issues very seriously, so the entire CentralPark.com dev team investigated immediately and our answer came only a few hours later, on Sunday, Sep 7 2014, 5:04pm:

Thank you for your report. We’ve checked the files and the database, plus the output itself, and could not find any malware or suspicious code. Could you provide more details so we can further investigate this?

And that was the entire conversation. Instead of replying and trying to back their malware report with details, they chose to write an article on their blog, with the following title: “No walk in the park: Centralpark.com malvertising highlights ongoing OpenX infection” (http://www.proofpoint.com/threatinsight/posts/no-walk-in-the-park.php)

The dev team has started another investigation and still couldn’t find anything suspicious.

First of all, their intro sentence “Proofpoint researchers have detected that the web site centralpark[.]com has been hit by malvertising and has been actively serving malware to visitors.”. If that was true, Google would have banned the website after only a few hours. But they didn’t, because there was no malware as we’ll explain below.

The ad server version was old indeed and we’ve taken steps to alleviate this. The zero day vulnerability (CVE-2013-7149) allows an attacker to execute arbitrary SQL code. This means they can only manipulate the database, not the files. We’ve thoroughly checked the database for patterns of known injections and we couldn’t find anything. But let’s move on to the next step in their investigation, maybe it offers more insight – we’ll skip over the stats; they’re correct, and OpenX is known to have a poor security standard.

Here’s a screenshot of where they claim the malware was injected. It’s pretty techy, but we’ll try to simplify it.
09082014-6img

 

The javascript code that serves the ads seems to have a prepended malicious script that points to sar[.]ttsselfstufy[.]com. We can trust the rest of the article which explains what malware is being pulled onto the client computer, however, the injected code can be there only if that code was in the database. The only place where this could’ve been in the database doesn’t allow anything to be written. It’s a “prepend” field and that has been disabled, exactly for this reason. In fact, our entire openx installation has been hardened as much as possible to withstand most of the common attacks. We’re not saying we’re 100% bulletproof, no one is, but chances for the ad server to be infected are quite low.

We also ran a scan on virustotal.com, in case we were missing something. The report can be viewed here:
https://www.virustotal.com/en/url/533d75b592d660e015ad3281ef15d6b7fc977a46d8f096cf7fc81dde7d02b78c/analysis/

Then we ran it on quttera.com too, since they reported suspicious files:
http://www.quttera.com/detailed_report/www.centralpark.com
Turns out their warnings are just a bug in the parser, probably because of our old js packer function.

We took a step even further and analyzed the requests using urlquery.net:
http://www.urlquery.net/report.php?id=1410462359637
Nothing here either.

We appreciate Proofpoint’s effort to notify us of a potential threat, and had their been an actual malvertising breach we would have worked to fix it immediately. However as the accusations are clearly inaccurate (and frankly slanderous) we trust they will do the right thing and pull the article from their blog immediately.

 

Security , ,

New Magento Minifier version

June 2nd, 2014

A new version of the Mangadreen Minifier is out, 1.0.4. Since the previous post there have been a few tweaks, fixes and performance improvements. The code has also been migrated to GitHub.

Here’s a direct link to the latest version (zipped): https://github.com/firewizard/Mandagreen_Minifier/archive/master.zip.

The plugin is currently in production on several Magento CE 1.5, 1.7, 1.8 and 1.9, so it should work on all Magento CE versions after 1.5.

Magento ,

Magento ‘Can’t get filling percentage’ memcache issue

May 30th, 2014

This is a known problem for anyone who runs Magento and uses memcache as a caching backend. But it’s actually not a Magento issue, it’s a Zend Framework problem, and it seems that even newer versions of ZF still have the same approach. I suspect this is actually a memcache problem, but even so, one should be able to handle things better in the code.

The idea is that memcached sometimes fails to report its extended stats, thus the “getExtendedStats” method returns false, when in fact it should return an array with all the servers. This should not be a problem though, instead of throwing an exception, why not retry with getStats and then just return 100 (which I guess it means the server is full). Here’s the revised code:

if ($memSize === null || $memUsed === null) {
    $mem = $this->_memcache->getstats();
    if (isset($mem['limit_maxbytes']) && $mem['limit_maxbytes'] > 0) {
        return ((int) (100 * ($mem['bytes'] / $mem['limit_maxbytes'])));
    } else {
        return 100;
    }
 
    //Zend_Cache::throwException('Can\'t get filling percentage');
}

You can get the entire file on github, but please note that this file corresponds to ZF v1.11.1 and Magento CE v1.5.1.0. It can definitely be adapted to any Magento version.

The easiest way to patch this without changing a core file (ie lib/Zend/Cache/Backend/Memcached.php) is to put the modified file in your app/code/local folder:
app/code/local/Zend/Cache/Backend/Memcached.php. since app/code/local comes first in the include path, the patched file will get included before the default one.

Disclaimer: This hasn’t been used in production, use at your own risk. In theory, it should be fine, but since it hasn’t passed any real testing, I don’t recommend doing so.

Magento , ,

Safari clear/float text flow issue

July 3rd, 2013

safari-bug
Here’s a strange and annoying Safari quirk I’ve stumbled upon yesterday, applies to both the Mac and Win versions. Basically, if there’s a text and two right floated elements (read images), and the first element is longer than the paragraph it precedes, and the next paragraph is also preceded by such a floated element, the result will be the text overlapping the floated element – see attached image. Here the code to replicate it: http://jsfiddle.net/TmJ9q/

If anyone knows any workarounds, just drop a comment. Others have reported this here, here and here, with the following test cases: http://jsfiddle.net/JwXke/28, http://jsfiddle.net/VaBCd/

 

CSS ,

Javascript version compare function

May 4th, 2013

Here’s quick javascript function for comparing versions. It works with any type of version strings, but it does not check for suffixes like ‘RC’, ‘alpha’, ‘stable’, etc. This was written in less than 10 minutes, so let me know if you find any bugs. Suggestions are welcomed anytime!

function versionCompare(a, b) {
	var A = a.split('.'), B = b.split('.'), ret = 0, base, sig, x, y;
	for(var i = 0, n = Math.max(A.length, B.length); i < n; i++) {
		x = 'undefined' == typeof A[i] ? 0 : parseInt(A[i]);
		y = 'undefined' == typeof B[i] ? 0 : parseInt(B[i]);
		
		base = Math.pow(10, n - i - 1);
		sig = 0;
		
		if( x < y ) { sig = -1; }
		else if( x > y ) { sig = 1; }
		
		ret += sig * base;
	}

	return ret == 0 ? 0 : ret / Math.abs(ret);
};

Javascript

Convert all MySQL tables and fields to UTF8 charset & collation

October 22nd, 2012

At some point in your web dev life you might face the “Illegal mix of collations” mysql error. Nothing complicated, you just need all your tables to use the same charset and collation and this can be fixed really easy using phpmyadmin or even from the command line. However, when the database has 300+ tables, doing it manually is really not an option, so it needs to be done using some kind of a script. It can be done in perl, php, python or anything for that matter, but I particularly liked this one line command, using bash & awk (always loved awk):

mysql --user=username --password=your_pass --database=dbname -B -N -e "SHOW TABLES"  | awk '{print "ALTER TABLE", $1, "CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;"}' | mysql --user=username --password=your_pass --database=dbname &

Found it on commandlinefu.com and worked like a charm.

Make sure to first read this article about converting charsets in mysql. Executing the command above might break things, depending on the database architecture. It’s always a good idea to create a backup of the database first.

MySQL ,

Magento memory leaks and custom options

August 30th, 2012

Most software, if not all, has memory leaks. Some are really bad at collecting garbage, other manage it better, but not enough. Magento had its problems since the beginning, and a lot of them have been fixed in the latest releases (using 1.6 to test this). However, it still has a long way until all major memory problems are gone. A simple test, like the one below, will reveal what I mean (put it in your magento root and run it from the CLI):

require "app/Mage.php";
Mage::app();

for($k = 0; $k < 100; $k++) {
    Mage::getModel('catalog/product')->load(123);
    echo $k . "\t" . (memory_get_usage()/1024/1024) . "\n";
}

This will show the current iteration and the memory usage. Replace “123” with a real product id – first, use a simple product, with no custom options. My result looks like this:

0 - 8.3790
...
99 - 8.3794

Looks good, very good actually.

Now try it with a product that has custom options. Not more than 5-6 options, but stuff a dozen values in one or two options. My test shows these numbers:

0 - 8.9398
...
99 - 17.5057

.

Along the way, you’ll see that the garbage collector kicks in and memory is freed, so max memory usage (for me) doesn’t exceed 20M. Cool, everything seems acceptable, and it seems that the old memory leaks have been fixed (read more over here, here and here ).

Now let’s try something different. Importing a few hundred products from a CSV. All products have custom options, usually the same, but with different values. The code could be summarized like this:

$options = $this->_getOptions($row);
$product = $this->initNewProduct(); //just inits the product with all the required data
$product->addData($specificFields)
        ->setAttributeSetId( $attributeSetId )
        ->setTypeId($typeId)
        ->setCategoryIds( array($someIds = 123) );
		
if( $options && count($options) ) {
    $product->setCanSaveCustomOptions(true);
    $product->setProductOptions($options);
}

$product->save();

To me, this looked fine and should have worked. I didn’t care it ran out of memory after 60-70 cycles, I could live with that. However, after approximately 20 hours of intense debugging, trying to figure out why the options aren’t showing up correctly on the frontend I decided to check the products in the admin. I had a huge surprise when I’ve noticed that for the second product, I had the first product’s options and the (correct) second product’s options. For the third, I had the first, second and third option sets. And so on, the last one was a complete mess. At some point Firefox couldn’t even load the interface anymore, there were simply too many options. Now that’s a memory leak for sure!

So, is there a fix? Seems like there was one, tested it for 3 products and it worked fine, no more leftover options. Initially, I was doing this:

$product->clearInstance();
unset($product);
				
gc_collect_cycles();

but it didn’t work, although gc_collect_cycles() helped keeping the memory growth under control.

So I added the following lines:

$product->getOptionInstance()->unsetOptions()->clearInstance();
unset($product, $options); //instead of just unset($product);

and voila – all custom options have been created correctly. Haven’t really analyzed why this did the trick and why it didn’t work without it. Either way, I’m happy the code works and I hope this will help someone else someday.

Magento , ,

The amazing world of Magento

July 5th, 2012

Magento has its positives and negatives, but this one is a must-share issue. While testing a payment gateway problem I discovered the following error (on Magento 1.7):

Item price:   33.15
Item qty:     1
Row subtotal: 33.15
---
Order Subtotal 33.15 - OK
Discount (fixed amount, applied before tax): 33.00 - OK
Tax (24%) 0.03 ( 0.24/1.24 * (33.15 - 33.00) ) - OK
Grand Total: 0.14 - NOT OK

Wow, so let’s see, subtotal 33.15 – discount 33.00 = grand total of 0.14. Not exactly correct, I would dare say.

Magento

Updated version of the Magento CSS & JS Minifier

May 31st, 2012

I’ve released the first version of the magento css&js minifier more than a year and half ago. Since then, it has been downloaded over 200 times – roughly 10 downloads per month. It’s a decent number, considering its utility and the fact that it hasn’t been promoted or included on Magento Connect. The first version was compatible with all Magento versions greater than 1.4. Starting with Magento 1.7, the extension failed to work on the admin pages, because of the -webkit-keyframes css declarations.

What’s new in version 1.0.1

  • Changed namespace from Oxygen to Mandagreen
  • Upgraded JsMin to version 1.1.2
  • Upgraded CssMin to version 3.0.1
  • Added backend options for converting HSL values, converting font-weight values, converting named colors and replacing variables

 

Download & Install

Download Minifier for Magento (31.03 kB), then unzip it and copy the app/ folder to your Magento root folder.

Logout from the admin if you’re already logged in, then log back in. Go to Cache Management, click on the “Flush cache storage” button, then go to Configuration > Developer and Enable javascript minifier and css minifier under CSS & JS Minifier. Also, make sure that “Merge JavaScript Files” and “Merge CSS Files” are enabled, under JavaScript Settings, and CSS Settings respectively. Go back to Cache Management and click the “Flush Javascript/CSS cache” button.

 

Upgrading from 0.1.0

If you’re already using the first version of the extension, you’ll need to remove the old files. Go to your magento root folder, then navigate to the etc/modules folder. Remove the file called Oxygen_Minifier.xml. Then, navigate to app/code/local and remove the Oxygen folder (or Oxygen/Minifier, if you have other modules with this namespace).

Next, issue the following mysql command, using the mysql console or PhpMyAdmin:

update core_config_data set path = replace(path, 'oxy_', 'mg') where path like '%oxy_minifier%';

You can now install the module as explained above.

 

Compatibility

This extension has been tested with all Magento CE versions 1.4 through 1.7.

 

Supporting the Module

If you enjoy using the Magento CSS & JS Minifier, help us improve it and make it even better. Since this project it entirely open source and free for everyone, we welcome any donations and consider them a sign of appreciation. Donate

We also appreciate your feedback, both positive and negative, as long as they are constructive.

If any of you know how to package an extension for Magento Connect, I would appreciate the help. I’ve given up after 2 hours or repeated failures.

Magento , , ,