Archive

Posts Tagged ‘Magento’

Magento memory leaks and custom options

August 30th, 2012

Most software, if not all, has memory leaks. Some are really bad at collecting garbage, other manage it better, but not enough. Magento had its problems since the beginning, and a lot of them have been fixed in the latest releases (using 1.6 to test this). However, it still has a long way until all major memory problems are gone. A simple test, like the one below, will reveal what I mean (put it in your magento root and run it from the CLI):

require "app/Mage.php";
Mage::app();

for($k = 0; $k < 100; $k++) {
    Mage::getModel('catalog/product')->load(123);
    echo $k . "\t" . (memory_get_usage()/1024/1024) . "\n";
}

This will show the current iteration and the memory usage. Replace “123″ with a real product id – first, use a simple product, with no custom options. My result looks like this:

0 - 8.3790
...
99 - 8.3794

Looks good, very good actually.

Now try it with a product that has custom options. Not more than 5-6 options, but stuff a dozen values in one or two options. My test shows these numbers:

0 - 8.9398
...
99 - 17.5057

.

Along the way, you’ll see that the garbage collector kicks in and memory is freed, so max memory usage (for me) doesn’t exceed 20M. Cool, everything seems acceptable, and it seems that the old memory leaks have been fixed (read more over here, here and here ).

Now let’s try something different. Importing a few hundred products from a CSV. All products have custom options, usually the same, but with different values. The code could be summarized like this:

$options = $this->_getOptions($row);
$product = $this->initNewProduct(); //just inits the product with all the required data
$product->addData($specificFields)
        ->setAttributeSetId( $attributeSetId )
        ->setTypeId($typeId)
        ->setCategoryIds( array($someIds = 123) );
		
if( $options && count($options) ) {
    $product->setCanSaveCustomOptions(true);
    $product->setProductOptions($options);
}

$product->save();

To me, this looked fine and should have worked. I didn’t care it ran out of memory after 60-70 cycles, I could live with that. However, after approximately 20 hours of intense debugging, trying to figure out why the options aren’t showing up correctly on the frontend I decided to check the products in the admin. I had a huge surprise when I’ve noticed that for the second product, I had the first product’s options and the (correct) second product’s options. For the third, I had the first, second and third option sets. And so on, the last one was a complete mess. At some point Firefox couldn’t even load the interface anymore, there were simply too many options. Now that’s a memory leak for sure!

So, is there a fix? Seems like there was one, tested it for 3 products and it worked fine, no more leftover options. Initially, I was doing this:

$product->clearInstance();
unset($product);
				
gc_collect_cycles();

but it didn’t work, although gc_collect_cycles() helped keeping the memory growth under control.

So I added the following lines:

$product->getOptionInstance()->unsetOptions()->clearInstance();
unset($product, $options); //instead of just unset($product);

and voila – all custom options have been created correctly. Haven’t really analyzed why this did the trick and why it didn’t work without it. Either way, I’m happy the code works and I hope this will help someone else someday.

Magento , ,

The amazing world of Magento

July 5th, 2012

Magento has its positives and negatives, but this one is a must-share issue. While testing a payment gateway problem I discovered the following error (on Magento 1.7):

Item price:   33.15
Item qty:     1
Row subtotal: 33.15
---
Order Subtotal 33.15 - OK
Discount (fixed amount, applied before tax): 33.00 - OK
Tax (24%) 0.03 ( 0.24/1.24 * (33.15 - 33.00) ) - OK
Grand Total: 0.14 - NOT OK

Wow, so let’s see, subtotal 33.15 – discount 33.00 = grand total of 0.14. Not exactly correct, I would dare say.

Magento

Updated version of the Magento CSS & JS Minifier

May 31st, 2012

I’ve released the first version of the magento css&js minifier more than a year and half ago. Since then, it has been downloaded over 200 times – roughly 10 downloads per month. It’s a decent number, considering its utility and the fact that it hasn’t been promoted or included on Magento Connect. The first version was compatible with all Magento versions greater than 1.4. Starting with Magento 1.7, the extension failed to work on the admin pages, because of the -webkit-keyframes css declarations.

What’s new in version 1.0.1

  • Changed namespace from Oxygen to Mandagreen
  • Upgraded JsMin to version 1.1.2
  • Upgraded CssMin to version 3.0.1
  • Added backend options for converting HSL values, converting font-weight values, converting named colors and replacing variables

 

Download & Install

Download Minifier for Magento (31.03 kB), then unzip it and copy the app/ folder to your Magento root folder.

Logout from the admin if you’re already logged in, then log back in. Go to Cache Management, click on the “Flush cache storage” button, then go to Configuration > Developer and Enable javascript minifier and css minifier under CSS & JS Minifier. Also, make sure that “Merge JavaScript Files” and “Merge CSS Files” are enabled, under JavaScript Settings, and CSS Settings respectively. Go back to Cache Management and click the “Flush Javascript/CSS cache” button.

 

Upgrading from 0.1.0

If you’re already using the first version of the extension, you’ll need to remove the old files. Go to your magento root folder, then navigate to the etc/modules folder. Remove the file called Oxygen_Minifier.xml. Then, navigate to app/code/local and remove the Oxygen folder (or Oxygen/Minifier, if you have other modules with this namespace).

Next, issue the following mysql command, using the mysql console or PhpMyAdmin:

update core_config_data set path = replace(path, 'oxy_', 'mg') where path like '%oxy_minifier%';

You can now install the module as explained above.

 

Compatibility

This extension has been tested with all Magento CE versions 1.4 through 1.7.

 

Supporting the Module

If you enjoy using the Magento CSS & JS Minifier, help us improve it and make it even better. Since this project it entirely open source and free for everyone, we welcome any donations and consider them a sign of appreciation. Donate

We also appreciate your feedback, both positive and negative, as long as they are constructive.

If any of you know how to package an extension for Magento Connect, I would appreciate the help. I’ve given up after 2 hours or repeated failures.

Magento , , ,

Workaround for attribute store labels not being displayed in Magento

February 12th, 2012

Here’s a quick workaround for a rather strange issue – sometimes, calling $attribute->getStoreLabel() returns null, although everything looks fine in the admin and in the database. It might be something that is slipping my mind, or it might be a bug, not really sure. However, I had to deal with it and come up with a fix, so I’d thought I’d share.

$_aIds = array();
foreach( $prod->getAttributes() as $attribute ) {
	//need the attribute_id which is missing from the attribute object
	if( !isset($_aIds[$attribute->getAttributeCode()]) ) {
		$_aIds[$attribute->getAttributeCode()] = $attribute->getIdByCode('catalog_product', $attribute->getAttributeCode());
	}
			
	//if there's not store_label try to set it
	if( !$attribute->getStoreLabel() ) {
		$labels = $attribute->getResource()->getStoreLabelsByAttributeId( $_aIds[$attribute->getAttributeCode()] );
		if( array_key_exists(Mage::app()->getStore()->getStoreId(), $labels)) {
			$attribute->setStoreLabel( $labels[Mage::app()->getStore()->getStoreId()] );
		} 
		else {
			$attribute->setStoreLabel( $attribute->getFrontendLabel() );
		}
	}
}
// end workaround

Applies to Magento 1.5.x

Magento ,

Patching the Magento USPS module

January 20th, 2011

Since their last update on Jan 3rd, 2011 USPS has stopped working on all of my Magento distro’s. The reason for that is a change in USPS’ response XML values. Magento has its part of blame here, first for not patching it immediately (which again shows how much they care about the CE users) and second for not coding it properly in the first place – they’re using the method name (i.e. Express Mail) to see if that method is allowed from the admin, instead of using the CLASSID attribute.

This issue has been reported on magentocommerce.com bug tracking system and it’s also discussed in the forum. USPS makes it clear that they’ll discontinue all API versions prior RateV4 ad IntlRateV2:
“All Rate Calculator API integrators are encouraged to migrate to the latest API versions (RateV4, IntlRateV2):
RateV4 and IntlRateV2 will be the only Rate Calculator API versions to offer the full range of new products and functionality
Rate, RateV2, RateV3 and IntlRate will be retired in May 2011, requiring all integrators to migrate to the latest versions”

As explained on the forums, the fix is quite simple, as USPS only added the &lt;sup&gt;&reg;&lt;/sup&gt; which is the encoded (twice :) ) version of the registered sign in a sup tag. Magento values are defined without those so when the code tries to match them it fails. All you have to do is clean that string and you’re good to go. I don’t like to modify core files so the fix involves copying app/code/core/Mage/Usa/Model/Shipping/Carrier/Usps.php to app/code/local/Mage/Usa/Model/Shipping/Carrier/ and working on the local version of Usps.php. Strip the special chars:

$mailService = str_replace('&reg;', '', strip_tags(htmlspecialchars_decode(htmlspecialchars_decode((string)$postage->MailService))));

and

$svcDescription = str_replace('&reg;', '', strip_tags(htmlspecialchars_decode(htmlspecialchars_decode((string)$service->SvcDescription))));

 

Here’s the archive with the patch. Just download it and unzip it in the root of your Magento distribution. Download USPS patch for Magento 1.4.2.0 (8.59 kB)

 

Note: This patch applies to Magento CE 1.4.2.0. For all other versions you will have to do it manually as explained above.

 

Magento, PHP , ,

Anatomy of a Magento extension

January 5th, 2011

Magento Commerce has been on the OSS “market” for a while now and I see more and more developers, designers and of course store owners migrating their ecommerce sites to Magento or installing it for their new stores. I’ve been coding on Magento since its beta 0.6 version, actually my first integration was based on this beta. I’ve been advised not to use the beta and they were right. However, I had my reasons for recommending it, using it and somehow advocating for that fresh, promising open source piece of software. In the meanwhile, it turned out I was right – it’s one of the most complete, scalable & feature-rich free ecommerce solution. Not the easiest to integrate or use, not the smoothest or lightweight, but it’s come a long way and it still have room for improvements.

One of the things I liked from the start (as a developer) was the ability to extend it in a very simple and loosely coupled way (unlike other solutions). Adding the OOP and design patterns made it perfect for my way of thinking. And I think this is the way to go for any major application, even if it might seem cluttered or with a steep learning curve at a first glance. I remember one of the first problems I faced was module creation – I was adding all my code in the core/Mage folder, I was overwriting a lot of the core code although I knew it was bad and so on. However, once I understood how to create a module, things became clearer and much simpler. So here’s a beginner’s guide to creating a new module. Although it’s meant for beginners, you should know the basic folder layout (skins, library, js, apps).

All Magento code resides in app/code and the lib folder. If you check app/code you’ll notice three subfolders: community, core, local. You should be adding your stuff in local or community (usually only when it is a community contribution). So let’s create our first module. I’ll use the namespace mandagreen and the module will be called HelloWorld. For this I need to create the following folders under app/code/local:
Mandagreen/HelloWorld

To keep things simple, in this example we’ll be creating one Block, one Helper and one Model – no controllers & no Resource Models. Here’s where each of these will reside:
blocks – app/code/local/Mandagreen/HelloWorld/Block
helpers – app/code/local/Mandagreen/HelloWorld/Helper
models – app/code/local/Mandagreen/HelloWorld/Model

Create these three folders and then also create a new etc folder. The structure will then look like this:

    app/code/local/Mandagreen/HelloWorld/Block
    app/code/local/Mandagreen/HelloWorld/etc
    app/code/local/Mandagreen/HelloWorld/Helper
    app/code/local/Mandagreen/HelloWorld/Model

The etc folder holds configuration information – in this case we’ll focus only on the module configuration file, which is config.xml. Let’s create it and start adding some xml into it:

<?xml version="1.0"?>
<config>
	<modules>
		<Mandagreen_HelloWorld>
			<version>0.0.1</version>
		</Mandagreen_HelloWorld>
	</modules>

	<global>
		<models>
			<helloworld>
				<class>Mandagreen_HelloWorld_Model</class>
			</helloworld>
		</models>
 
		<blocks>
			<helloworld>
				<class>Mandagreen_HelloWorld_Block</class>
			</helloworld>
		</blocks>

		<helpers>
			<helloworld>
				<class>Mandagreen_HelloWorld_Helper</class>
			</helloworld>
		</helpers>
	</global>

	<frontend>
		<layout>
			<updates>
				<helloworld>
					<file>mg_helloworld.xml</file>
				</helloworld>
			</updates>
		</layout>

		<translate>
			<modules>
				<Mandagreen_HelloWorld>
					<files>
						<default>Mandagreen_HelloWorld.csv</default>
					</files>
				</Mandagreen_HelloWorld>
			</modules>
		</translate>
	</frontend>
</config>

That’s a lot, isn’t it? :) No worries, I’ll explain. Under the main node (config), youll usually need the global node. The modules node is being used to hold mode info about the current module, maybe dependencies and so on. The frontend node is usually used for defining layout handlers and translation files, but they can do more than that. In this example, we’re gonna have our own layout file called mg_helloworld.xml – you can use any name on it (as long as it’s unique), but I prefer to namespace it as well. Also, I’ve defined a translation file Mandagreen_HelloWorld.csv which needs to be created in app/locale/en_US (or whatever your default locale is).

The global node defines all the classes you’ll be using, in this case helpers, models and blocks. You can see that they are defined the same way, but in different locations: <helpers />, <models /> and <blocks />. Let’s use the helpers node as an example.

		<helpers>
			<helloworld>
				<class>Mandagreen_HelloWorld_Helper</class>
			</helloworld>
		</helpers>

The helloworld node defines a handle and you should be careful to always use unique names. If you’re not sure if a certain name exist, namespace it (mghelloworld for example). This handle will be used to instantiate models, call helpers or use blocks using the Magento factory. For example, for models, you’ll be using Mage::getModel('helloworld/modelname'); instead of new Mandagreen_HelloWorld_Model_Modelname();. Same with helpers, inside a block template – $this->helper('helloworld')->someMethod();.
The other node, class is being used to define the namespace of the class name. In the example above, the name of the Modelname class has to being with Mandagreen_HelloWorld_Helper. A few examples might help:

  • Mandagreen_HelloWorld_Model_Salute, will be found in Mandagreen/HelloWorld/Model/Salute.php and can be accessed via Mage::getModel('helloworld/salute')
  • Mandagreen_HelloWorld_Model_Salute_Hi, will be found in Mandagreen/HelloWorld/Model/Salute/Hi.php and can be accessed via Mage::getModel('helloworld/salute_hi')
  • Mandagreen_HelloWorld_Helper_Help, will be found in Mandagreen/HelloWorld/Model/Help.php and can be accessed via Mage::helper('helloworld/help')

Now that we explained the config file, let’s move on to actually creating the classes and code. I’ll use the example above and create the following files:

Mandagreen/HelloWorld/Block/Standard.php

<?php

class Mandagreen_HelloWorld_Block_Standard extends Mage_Core_Block_Template {
    function getSomething() {
          return Mage::getModel('helloworld/salute')->getName();
    }
}

Mandagreen/HelloWorld/Helper/Help.php

<?php

class Mandagreen_HelloWorld_Helper_Help extends Mage_Core_Helper_Abstract {
    function shouldSayHi() {
          return true;
    }
}

Mandagreen/HelloWorld/Model/Salute.php

<?php

class Mandagreen_HelloWorld_Model_Salute extends Mage_Core_Model_Abstract { //or Varien_Object or none
    function getName() {
          //do some heavy logic here
          return 'John';
    }
}

One more file is required for the translations to work with this module – a “default” Data helper, defined in Mandagreen/HelloWorld/Helper/Data.php like this:

class Mandagreen_HelloWorld_Helper_Data extends Mage_Core_Helper_Abstract {}

We have the classes but we also need to use them in a template, so let’s create standard.phtml in app/design/frontend/default/default/template/helloworld (create all additional folders if needed). Also, create mg_helloworld.xml under app/design/frontend/default/default/layout.

For the template, things are very easy:

<div style="background: red; padding: 20px;">
	<?php if( $this->helper('helloworld/help')->shouldSayHi() ): ?>
	Hello <?php echo $this->getSomething(); ?>
	<?php else: ?>
	Can't say anything...
	<?php endif; ?>
</div>

while for the layout we’ll use this approach:

<?xml version="1.0"?>
<layout version="0.1.0">
    <default>
        <reference name="content">
            <block type="helloworld/standard" name="helloworld" template="helloworld/standard.phtml" after="-" />
        </reference>
    </default>
</layout>

This should display “Hello John” at the end of the content area on most of the pages, including the homepage. But wait, before you try that you’ll have to enable the module in app/etc/modules. Create a file called Mandagreen_HelloWorld.xml with the following code:

<?xml version="1.0"?>
<config>
    <modules>
        <Mandagreen_HelloWorld>
            <active>true</active>
            <codePool>local</codePool>
        </Mandagreen_HelloWorld>
    </modules>
</config>

Clear all magento cache and refresh the page – your first module, up and running! And here’s the archive for this tutorial – Download Hello World Magento Module (2.99 kB)

Further/recommended reads:

Magento, Uncategorized

Adding Customer Comments as Order Status Comments using Magento & OneStepCheckout

October 24th, 2010

A few days ago I got an email from someone asking me about adding the default customer comments in OneStepCheckout (www.onestepcheckout.com) as regular order comments. I thought I’d share this quick & simple hack with everyone, so here’s what you have to do:
Open app/code/local/Idev/OneStepCheckout/Helper/Data.php, and after

$observer->getEvent()->getOrder()->setOnestepcheckoutCustomercomment($orderComment);

add this line:
$observer->getEvent()->getOrder()->setState( Mage_Sales_Model_Order::STATE_NEW, true, $orderComment, false );

$observer->getEvent()->getOrder()->setState( 
    $observer->getEvent()->getOrder()->getStatus(), 
    true, 
    $orderComment, 
    false 
);

This will add the comments on the regular comments/statuses thread, as well as a customer comment. If you want to disable customer comments, just comment the original code.

Actually, this approach should also work with the out-of-the-box Magento one step checkout. Add an input on the last step, create a listener for

checkout_type_onepage_save_order

and use the same piece of code:

$observer->getEvent()->getOrder()->setState( $observer->getEvent()->getOrder()->getStatus(), true, $this->_getRequest()->getPost('order_comments'), false );

Note: Tested on Magento 1.4.1.0 & 1.4.1.1

Magento ,

Introducing Magento CSS & JS Minifier

September 15th, 2010

As the title says, I’m glad to announce my first public Magento extension (not yet added in the Connect repository). During my 3 years experience with Magento, I’ve worked on a lot of custom extensions, improvements & fixes, but most of them were client-specific, plus they weren’t designed to have a backend interface (with a few exceptions). This one, however, is entirely configurable from the Admin and it’s both simple and effective.

 

What it does

This quick optimizer parses all javascript & css files included on a page and removes all unnecessary characters. The most simple step is to remove spaces, tabs and new lines – but there’s more than just that. Of course, for small files compression is insignificant, but when you work with almost 600KB and around 30 requests, you can save a lot. Here’s a quick math on one of my Magento installs:

Javascript – 26 requests, 479 KB
CSS – 4 requests, 102 KB

With Magento’s default merging enabled:
Javascript – 1 request, 360 KB (not sure why this is smaller then the 26 summed up, but nvm)
CSS – 1 request, 108.2 KB (same for this one too, but again, nvm)

We’ve already saved 28 requests, which means less overhead – quicker download times for user, less stress on the server.

With the Minifier enabled:
Javascript is 255 KB, which means almost 47% compression
CSS is 92 KB, which means almost 10% compression

 

Advantages

 

Disatvantages

  • Need to write javascripts very careful, adding a semicolon after almost everything
  • Have to rewrite of Mage_Core_Model file
  • Have to override two Magento methods
  • Additional processing time (insignificant in my opinion)

 

Credits

This plugin wouldn’t exist if it weren’t for these two outstanding PHP projects:
Joe Scylla’s CssMin
Ryan Grove’s JsMin
Big thanks to both of them.

 

Download & Install


First, Download Minifier for Magento (31.03 kB), then unzip it and copy the app/ folder to your Magento root folder.

Logout from the admin if you’re already logged in, then login. Go to Cache Management, click on the “Flush cache storage” button, then go to Configuration > Developer and enable all the options, as shown in the attached screenshot. Go back to Cache Management and this time click on the “Flush Javascript/CSS cache”.

Go to your store frontend and behold, you’re now using compressed js’s and css’s.

Works on Magento 1.4+

Updated on May 31st, 2012 – all download links in this page refer to the latest version of the plugin. Some explanations on this post might be inaccurate. For the latest details please check new post called Updated version of the Magento CSS & JS Minifier

Magento , , ,

How to export customers from orders between certain dates in Magento

August 23rd, 2010

Last week a client who runs his store on Magento Commerce asked me how he could export a list of customers who purchased between certain dates in August. Of course, I first went to the Administration area and tried several approaches (order reports, customer reports, data export). Unfortunately nothing worked, so I had to write a script or a plain SQL and run it from the CLI. I preferred a straight query, rather than a script, and I tried not to use sub-selects. Here’s the adapted part, where I’ve added some variables to keep all the attribute id’s:

SET @etID := (SELECT entity_type_id FROM eav_entity_type WHERE entity_type_code = 'order_address');
SET @atFn := (SELECT attribute_id FROM eav_attribute WHERE attribute_code = 'firstname' AND entity_type_id = @etID);
SET @atLn := (SELECT attribute_id FROM eav_attribute WHERE attribute_code = 'lastname' AND entity_type_id = @etID);
SET @atEmail := (SELECT attribute_id FROM eav_attribute WHERE attribute_code = 'customer_email');

SET @startDate := '2010-08-06 00:00:00';
SET @endDate := '2010-08-13 23:59:59';

SELECT o.increment_id, e.value as email, ln.value as lastname, fn.value as firstname
FROM sales_order o
INNER JOIN sales_order_varchar e ON e.entity_id = o.entity_id AND attribute_id = @atEmail
INNER JOIN sales_order_entity lne ON lne.parent_id = o.entity_id AND lne.entity_type_id = @etID
INNER JOIN sales_order_entity_varchar ln ON ln.entity_id = lne.entity_id AND ln.attribute_id = @atLn
INNER JOIN sales_order_entity_varchar fn ON fn.entity_id = lne.entity_id AND fn.attribute_id = @atFn
WHERE o.created_at BETWEEN @startDate AND @endDate
GROUP BY o.increment_id
ORDER BY o.increment_id DESC;

The only problem is when the Shipping Name is different than the Billing Name – there’s no control over that, as we’re grouping by increment_id. This could easily be fixed in a PHP script, but the approach there should be different (using collections).

Hope this helps anyone, as it did the trick for me.

Solution tested under Magento 1.4.0.1

Magento, MySQL ,

Fixing “Catalog Price Rules” cart issue in Magento 1.4.1.0

June 29th, 2010

After recently upgrading one of my stores running Magento Commerce I found out the the latest version (at this time 1.4.1.0) had a major bug. Don’t want to sound too harsh, but unfortunately Magento’s support was again awful – there have been one thread on the forum and one issue in their bug tracking system for a while now (almost two weeks if not more) and still no update, although as I said the bug has quite an impact – even tweet’ed the problem to @magento and haven’t gotten any reply (again it’s not the first time). Yes, the product is free, yes it’s open source, but I think a bit of transparency and better communication could come to their advantage.

Bug details/behavior

So, updated from 1.4.0.1 to 1.4.1.0, everything looks fine, catalog price rules are being applied in the catalog (categories and product pages), but when adding the product to the cart the quote item price was the regular price not the special one.

Solution/Fix

After a few hours playing with the rules, observer and other core elements I found the problem. A big “thank you” goes to “myself” who posted the second comment for the issue mentioned above. The problem was within the CatalogRule Observer, when fetching the ID of the Customer Group. Here’s how to fix it:

1 – create the following folders in your Magento distro: app/code/local/Mage/CatalogRule/Model
2 – copy app/code/core/Mage/CatalogRule/Model/Observer.php to app/code/local/Mage/CatalogRule/Model
3 – open the new/copied file and go to line 105. Change this code:

        if ($observer->hasCustomerGroupId()) {
            $gId = $observer->getEvent()->getCustomerGroupId();
        } elseif ($product->hasCustomerGroupId()) {
            $gId = $product->hasCustomerGroupId();
        } else {
            $gId = Mage::getSingleton('customer/session')->getCustomerGroupId();
        }

to:

        if ($observer->hasCustomerGroupId()) {
            $gId = $observer->getEvent()->getCustomerGroupId();
        } elseif ($product->hasCustomerGroupId()) {
            $gId = $product->getCustomerGroupId();
        } else {
            $gId = Mage::getSingleton('customer/session')->getCustomerGroupId();
        }

To be more precise, you have to change hasCustomerGroupId to getCustomerGroupId on line 105.

You can now enjoy your store again!

Magento