On Sunday, Sep 7 2014, 11:10am CentralPark.com received a message from Wayne Huang (whuang@proofpoint.com) from proofpoint.com informing us that we’ve been hit by malware. To be more precise he was saying our Ad Server had been compromised. Here’s the entire message we received:

I’m writing to inform you that your website has been hit by malware. Your OpenX ad server, used to serve out the ads on the upper-left corner of your website, has been infected.
Specifically, it is this URL: [url removed for security reasons]
Please react asap. If you need help you are welcome to reach out to me at whuang@proofpoint.com.

We take security issues very seriously, so the entire CentralPark.com dev team investigated immediately and our answer came only a few hours later, on Sunday, Sep 7 2014, 5:04pm:

Thank you for your report. We’ve checked the files and the database, plus the output itself, and could not find any malware or suspicious code. Could you provide more details so we can further investigate this?

And that was the entire conversation. Instead of replying and trying to back their malware report with details, they chose to write an article on their blog, with the following title: “No walk in the park: Centralpark.com malvertising highlights ongoing OpenX infection” (http://www.proofpoint.com/threatinsight/posts/no-walk-in-the-park.php)

The dev team has started another investigation and still couldn’t find anything suspicious.

First of all, their intro sentence “Proofpoint researchers have detected that the web site centralpark[.]com has been hit by malvertising and has been actively serving malware to visitors.”. If that was true, Google would have banned the website after only a few hours. But they didn’t, because there was no malware as we’ll explain below.

The ad server version was old indeed and we’ve taken steps to alleviate this. The zero day vulnerability (CVE-2013-7149) allows an attacker to execute arbitrary SQL code. This means they can only manipulate the database, not the files. We’ve thoroughly checked the database for patterns of known injections and we couldn’t find anything. But let’s move on to the next step in their investigation, maybe it offers more insight – we’ll skip over the stats; they’re correct, and OpenX is known to have a poor security standard.

Here’s a screenshot of where they claim the malware was injected. It’s pretty techy, but we’ll try to simplify it.

The javascript code that serves the ads seems to have a prepended malicious script that points to sar[.]ttsselfstufy[.]com. We can trust the rest of the article which explains what malware is being pulled onto the client computer, however, the injected code can be there only if that code was in the database. The only place where this could’ve been in the database doesn’t allow anything to be written. It’s a “prepend” field and that has been disabled, exactly for this reason. In fact, our entire openx installation has been hardened as much as possible to withstand most of the common attacks. We’re not saying we’re 100% bulletproof, no one is, but chances for the ad server to be infected are quite low.

We also ran a scan on virustotal.com, in case we were missing something. The report can be viewed here:
https://www.virustotal.com/en/url/533d75b592d660e015ad3281ef15d6b7fc977a46d8f096cf7fc81dde7d02b78c/analysis/

Then we ran it on quttera.com too, since they reported suspicious files:
http://www.quttera.com/detailed_report/www.centralpark.com
Turns out their warnings are just a bug in the parser, probably because of our old js packer function.

We took a step even further and analyzed the requests using urlquery.net:
http://www.urlquery.net/report.php?id=1410462359637
Nothing here either.

We appreciate Proofpoint’s effort to notify us of a potential threat, and had their been an actual malvertising breach we would have worked to fix it immediately. However as the accusations are clearly inaccurate (and frankly slanderous) we trust they will do the right thing and pull the article from their blog immediately.

Cristi Security malvertising, openx, security

There are times when your website or application might get hacked. The usual reasons include weak passwords, outdated software or poor code (especially on validating user input). The same apply to OpenX – your ad server can be hacked due to an outdated version of OpenX, weak passwords, insecure server settings or a combinations of the above. Most common hacks add an iframe or script to the banner page distributing malware (a virus, trojan horse, or similar). This can get your website (the one that uses your infected openx server) to be blocked on all major browsers and most of the search engines.

To decrease the chances of having your openx hijacked, it’s recommended to follow these quick and easy steps. You’ll notice that some of them apply to any web application, like using strong passwords (at least 6 chars, using both uppercase and lowercase letters, digits and special chars).

Things to do with your OpenX installation
1. Always update the application and plugins to the latest versions. This ensures your ad server has the latest patches and security fixes, but it also makes hacking a bit more difficult. You can check if you have the latest version by going to Configuration / Product Updates in the OpenX administration interface.

2. If your administrator username is admin or administrator (or anything similar), change it to something less common, like your first name or last name. This cannot be done from the openx admin interface, but it can be done directly in mysql (using phpmyadmin for instance) – the table you are looking for is [table_prefix]_users

3. If there are other users who manage the inventory, create separate accounts for them and make them advertisers, not admins. Keep the admin account safe and do not share it with anyone.

4. If you only server banners on your website(s), you can remove all users from Advertisers and Websites. As a general rule, when you create a new advertiser, do not also create a username. If you don’t offer OpenX access to advertisers, there’s really no point in adding new users.

5. Always use the logout link to log out, instead of just closing the tab or window. This prevents session hijacking or keeping an active session on a public computer.

6. Disable all plugins that you don’t use

Things to do on the server side
1. Setup the openx database with its own username and use a really strong password. Make sure the user is only allowed to connect from the server that runs openx (where your web server is located) – this is usually the same machine, so localhost will be just fine. The password should be really strong. You can use an online password generator tool, like this one: http://www.pctools.com/guides/password/

2. Add an extra layer of authentication, using http basic auth for example. Follow the instruction in the links below:
http://httpd.apache.org/docs/2.0/howto/auth.html
http://httpd.apache.org/docs/2.0/programs/htpasswd.html

3. Change the database structure of the banner’s table so it doesn’t allow append & prepend codes anymore. You can do this by changing the two fields in the banners table ([table_prefix]_banners) from varchar(255) to varchar(0) (through phpmyadmin or the cli):
alter table openx_banners change append append varchar(0);
alter table openx_banners change prepend prepend varchar(0);

3a. Change the database structure of the zone’s table so it doesn’t allow append & prepend codes anymore – thanks Stefan for suggesting this.
alter table openx_zones change append append varchar(0);
alter table openx_zones change prepend prepend varchar(0);
Make sure to replace “openx” with the table prefix you use.

4. Set the proper permissions to all files & folders – all files should not be writable by the server by default, except the var/, plugins/ and www/images/ folders.
Your configuration file, located in var/, should not be writable:
chmod 0444 [domain].conf.php

5. Remove all installation files

What if I am already infected?
It’s important to find out all infected files or database entries and then figure out how they got there. Do not remove anything befire understanding how you got infected in the first place. You can disable the openx server temporarily (by adding this line in an .htaccess file in the folder www/delivery: deny from all).
Once you know where the security breach is, fix it, harden the server, change passwords (just to be sure), clean up the infected data and remove the deny all line from the .htaccess file.

We can also help you cleaning, securing and upgrading your openx installation(s) – contact us.

Cristi Security openx, security