## Friday, October 24, 2008

### Stop undeliverable spoofed spam using maildrop and PHP

A common problem among email providers is the growing practice of spammers using legitimate email addresses in the From: header of their messages, which means that unsuspecting users get flooded with potentially thousands of undeliverable messages due to the fact that many of the To: addresses on the spammers' lists are no longer valid.
A few of our customers had this issue, so I devised a quick way to filter out the undeliverable messages via Maildrop and php.

1. Set Postfix to pass mail through Maildrop

After installing maildrop (it should be readily available in your *nix distro's packaging system) Make sure Postfix is set up to pass mail to Maildrop by putting the following line in your master.cf file:
maildrop unix - n n - - pipeflags=DRhu user=vmail argv=/usr/local/bin/maildrop -d ${recipient}${recipient}

2. Place the following code in /usr/local/etc/mail/maildrop_filters.php
ini_set('display_errors', 'off');error_reporting(0);//---------------------------------------$recipients = array( 'recipient1@domain1.com', 'recipient2@domain2.com');$bad_subjects = array(        'Undelivered Mail',        'Delivery Status Notification',        'Undeliverable mail',        'Returned Mail',        'Delayed Mail',        'Delivery Failure',        'Warning: could not send message',        'Warning: message');//---------------------------------------// Don't edit below here//---------------------------------------$sender_address =$argv[1];$recipient_address =$argv[2];$stdin = fopen('php://stdin', 'r');$msg = '';$header = '';while ($buf = fread($stdin, 500) ) {$msg .= $buf;}$msg_lines = explode("\n", $msg);if ( is_array($msg_lines) ) {        foreach( $msg_lines as$cur_line ) {                $cur_line = trim($cur_line);                if ( $cur_line == '' ) { break; }$header .= $cur_line . "\n"; }}foreach($recipients as $recipient ) { if ( preg_match('/^To\:\s*\ foreach($bad_subjects as $subj ) { if ( preg_match("/^Subject:\s*" . preg_quote($subj) . "/im", $header) ) { exit(1); } } }}echo$msg;exit(0);?>

3. Add the following to your maildroprc file, which will probably be in /etc, /etc/mail, or /usr/local/etc/:

    RECIPIENT_ADDRESS="$1" exception { xfilter "/usr/local/bin/php /usr/local/etc/mail/maildrop_filters.php \"${SENDER}\" \"${RECIPIENT_ADDRESS}\"" } if ($RETURNCODE == 1 )    {        log "${OUTER_INDENT} Message for${RECIPIENT_ADDRESS} discarded. Returncode was ${RETURNCODE}" EXITCODE=0 exit } 4. Edit the$recipients array

Go back into /usr/local/etc/mail/maildrop_filters.php and edit the $recipients array as necessary - this is where you put the addresses that are being bombarded with the undeliverable mail messages: $recipients = array(                'recipient1@domain1.com',                'recipient2@domain2.com',                'recipient3@domain3.com',);

Additionally, you can add or remove subjects from the $bad_subjects array as required. Note that the filter matches subjects that *start* with the items listed in$bad_subjects, so "Undeliverable:" will match "Undeliverable: Mailbox not found" and "Undeliverable: Mailbox is full", and so on.

## Wednesday, October 22, 2008

### SwiftFile - Send files securely on the web

As the latest in a series of webtools that Context is launching, we've decided to offer an extremely easy, free, secure solution for uploading and sending files. SwiftFile.net offers a simple interface to upload and password protect files, making them unreadable even to the site administrator. Our goal was not to compete with filesharing giants like sendspace, mediafire, or rapidshare, but rather to offer a way to send sensitive documents very quickly and very securely. You can read the SwiftFile FAQ for information about how the data is stored and encrypted. Give it a try - it's fast, easy, and free. Additionally, thanks to the Fuse PHP framework, the site was put together from start to finish in 2 modest work days. We're hoping that a side effect of these tools will be to showcase the versatility and ease of use of the framework, which is now gaining ground in the PHP MVC community.

## Sunday, October 19, 2008

### suphp directory not owned by user

Installing suPHP on Apache 2 tonight, I was surprised that it was complaining about the parent directory for my document root not being owned by the specific user who owned the scripts. I had it owned by the apache user, www, and it turns out it (the parent directory which, for me, was /home/www) has to be owned by root in order for suPHP to traverse it.

## Thursday, September 04, 2008

### Why you should do your PHP development in FUSE

Let's face it - there are a lot of development frameworks out there, especially for PHP. CodeIgnitor and CakePHP come to mind immediately as leading PHP frameworks, but in this article I'm going to give some compelling reasons as to why you should be doing your PHP development with the Fuse Framework.

Fuse is a Model/View/Controller framework for PHP. If you're not familiar with MVC architecture, you may want to start with my other post, MVC and you: Partners in Freedom

1. FUSE is easy to get running

The installation scripts included with Fuse guarantee that you'll have a working install a few minutes after downloading. To get an idea of just how easy it is to get started with a Fuse project, have a look at the video I've posted here .
Fuse was built so that PHP developers don't have to feel like they were learning an entirely different language in order to make use of the framework. The syntax and structure follow normal PHP conventions, and everything from the method names to parameter order was designed to be as intuitive as possible.
Let's face it - everyone wants to use the newest, best, and most appropriate tool for the job. However, developers will often shy away from doing things differently because of the learning curve. They fear that introducing new technologies or methodologies will increase their project timeframe and decrease productivity. Fuse was built with a deadline-oriented mentality in mind: you can get started quickly, and you don't have to know every in and out of the framework to start building your project.

2. Data access has never been easier

Fuse's data modeling makes accessing your data easier than you ever thought possible. The management scripts will give you create, read, update, and delete access right off the bat, and customizing your queries is as simple as can be. Let's say you want to list your products, but also include their category_name, which lives in the product_categories table. In Fuse, all you have to do is open your ProductController and add this line:

public $list_options = array( 'include' => 'product_categories' ); That's it. Now when you iterate through your products, you can use the variable <{category_name}> in your template to display the category name for your product. Let's say we only want to display 20 categories? Try this: public$list_options = array( 'include' => 'product_categories', 'limit' => 20 );

Fuse offers a whole slew of data methods just like these, and your queries can be as simple or as complicated as you need them to be. You will rarely have to write a query, but if you want to, Fuse even offers an object to take the headache out of writing queries . And, as always, if you just want to hand code a query the old fashioned way, Fuse will never stop you. A core principal of Fuse is that it's designed to aid the programmer, not force him or her into the methodologies that we've deemed best.

3. A simple but extremely powerful templating engine

Fuse contains its own robust, intuitive templating engine that allows you to truly separate your code from your presentation. As with everything else, the templating system was designed to be intuitive, and "designer friendly". Want to loop through the products we fetched above? Try this:

<{ITERATOR products}>name: <{name}><br />category: <{category_name}><br /><br /><{/ITERATOR}>

Want to apply a function to one of your fields? How about we only display the first 200 characters of our description:
<{ITERATOR products}>name: <{name}><br />category: <{category_name}><br />description: <{ print(substr(description, 0, 200)) }><br /><{/ITERATOR}>

Maybe our products can be in more than one category, and we want to fetch them?

After adding a one line method to our Product model that looks like this:
public function get_categories() {return $this->product_categories->fetch_all();} We can do: <{ITERATOR products}>Name: <{name}>Categories:<{ITERATOR get_categories()}> <{name}><{/ITERATOR}><{/ITERATOR}> Yes, <{name}> will know that when you're in the products loop, you want the product name and when you're in the product categories loop, you want the category name. 4. Searching your data has definitely never been easier. Your client tells you that they want to be able to search records with any combination of title, date, description, id, and author. You know this is going to be an annoying task of using if/then statements to build a search query. Unless you're using Fuse, where you can do it all with a few lines of code. The FuseDataController object, which handles all of the wheeling and dealing your app does with the database, has builtin search capabilities that will automatically sanitize data, generate the search query, and maintain the search session across pages. All you have to do is tell it which fields a user can search on, and it's ready to go. You can search fields values directly, with wildcards on either side, between two date intervals, or even have Fuse automatically parse strings like "restaurant AND (mexican OR Thai)" simply by setting your 'filter_type' to 'parsed_boolean'. The search capabilities are discussed here 5. Builtin management of photos for albums, user profiles, etc. Let's say you're building a site that allows users to create a profile, add photo albums, then add photos to those albums. You're going to want several different sizes - a full sized image, a thumbnail for browsing, and a tiny thumbnail for a contact sheet type of display. You also want to watermark each image with your site's logo. This can be a pretty tedious task if you're not using the FusePhotoController, which should have you up and running in about a half hour if you're slow. 6. Simple but fully featured, scalable user management and ACL A lot of sites need user authentication. I've seen a lot of custom implementation in my time, and because of the complexity of doing it right, mostly they rely on a very basic user scheme that allows little or no granularity or scalability when it comes to setting permissions. Fuse has a simple to implement, but fully granular and scalable user authentication and permissions scheme built right in. Need to associate users with groups that all have different permissions? No problem. Have a user who's in the "editors" group, but shouldn't have access to delete an article? Just add a restriction for that user. Need to set it up so that when a user tries to access a restricted page, they are asked to login, then redirected back to the original page on success? It's already done. Password encryption? SHA-1, MD5, and crypt() are all supported. I could go on. You can read about it on your own here 7. Integration with existing non-Fused projects I mentioned above that Fuse endeavors to never prevent the developer from doing what he or she needs to do. If you have a project that's already done in standard, inline PHP, Fuse can go right alongside your existing code without upsetting any of the existing functionality. In fact, if you include the Fuse bootstrap in one of your existing php scripts, you can add Fuse functionality to that script without having to edit any of the existing code. I have several projects right now that were handed to me as inline PHP, and I put Fuse right on top of it without having to re-code a single line of the existing project. However, I can now use Fuse moving forward for new features and updated functionality. Conclusion There you have it. Just a selected list of reasons you should switch to Fuse. Today. For the project you're working on. Right now. Make things easier on yourself. I've introduced a lot of people to MVC development and Fuse and, without fail, every single one has said, after just one project, that they could never possibly go back to their old methodologies of inline scripting and manually writing every query. Comments welcome. ## Thursday, August 28, 2008 ### exchange defragment stuck or frozen I needed to defragment an Exchange 2000 mailbox store tonight because it had reached the 16GB limit. I started the process, let it run to about 5%, then walked away for about an hour. I came back, and it hadn't moved. I thought, "how long does it take to defragment the mailbox store??". I checked the temp files that the defragmenter was generating and noticed they hadn't been modified in an hour. Long story short: for some reason, clicking the mouse in the command window where the defragmenter is running will cause it to pause, and you can restart it by hitting F5. Thanks to the following link for the F5 info: http://www.eggheadcafe.com/software/aspnet/31768198/how-do-i-know-how-long-th.aspx ## Thursday, August 07, 2008 ### CS-Cart images not displayed after server move / mysqldump if you are using CS-Cart and you've copied the database using a mysqldump ,the binary data for the images may not have been migrated properly. You need to make sure to use the --hex-blob switch when dumping the database. More information on the --hex-blob switch can be found here ## Wednesday, August 06, 2008 ### ExpressionEngine flv upload file type not allowed To allow FLV files to be uploaded in ExpressionEngine without getting the file type not allowed message, add the following to /system/lib/mimes.php in the$mimes array:

'flv' => 'video/x-flv'

Usage:
    try {       require_once('PayFlowTransaction.class.php');//assumes it's in the current dir   $txn = new PayflowTransaction(); // //these are provided by your payflow reseller //$txn->PARTNER = 'yourpartnername';   $txn->USER = 'yourusername';$txn->PWD= 'yourpassword';   $txn->VENDOR =$txn->USER; //or your vendor name     //   // transaction information   //   $txn->TENDER = 'C'; //sets to a cc transaction$txn->ACCT = '4111111111111111'; //cc number   $txn->TRXTYPE = 'S'; //txn type: sale$txn->AMT = 1.00; //amount: 1 dollar   $txn->EXPDATE='0210'; //4 digit expiration date$txn->FIRSTNAME = 'Joe';   $txn->LASTNAME = 'Junior Shabadu';$txn->STREET = '123 mystreet';   $txn->CITY = 'Philadelphia';$txn->STATE = 'PA';   $txn->ZIP = '19115';$txn->COUNTRY = 'US';     //$txn->debug = true; //uncomment to see debugging information //$txn->avs_addr_required = 1; //set to 1 to enable AVS address checking, 2 to force "Y" response   //$txn->avs_zip_required = 1; //set to 1 to enable AVS zip code checking, 2 to force "Y" response //$txn->cvv2_required = 1; //set to 1 to enable cvv2 checking, 2 to force "Y" response   //$txn->fraud_protection = true; //uncomment to enable fraud protection$txn->process();      echo "success: " . $txn->txn_successful; echo "response was: " . print_r($txn->response_arr, true );     }  catch( TransactionDataException $tde ) { echo 'bad transaction data ' .$tde->getMessage();        }        catch( InvalidCredentialsException $e ) { echo 'Invalid credentials'; } catch( InvalidResponseCodeException$irc ) {   echo 'bad response code: ' . $irc->getMessage(); } catch( AVSException$avse ) {   echo 'AVS error: ' . $avse->getMessage(); } catch( CVV2Exception$cvve ) {   echo 'CVV2 error: ' . $cvve->getMessage(); } catch( FraudProtectionException$fpe ) {   echo 'Fraud Protection error: ' . $fpe->getMessage(); } catch( Exception$e ) {   echo $e->getMessage(); }  You will probably want to implement better error checking than what I did in the example. However, you'll still want to display TransactionDataException messages back to the user, because they're usually invalid expiration dates or something similar. ## Tuesday, July 15, 2008 ### jQuery isChildOf - is element a child of / ancestor of element - plugin If anyone knows of an existing jQuery method of determining whether an element is a child of another element, please feel free to post. I couldn't find one, so I developed the plugin below: (function($) {    $.fn.extend({ isChildOf: function( filter_string ) { var parents =$(this).parents().get();                   for ( j = 0; j < parents.length; j++ ) {           if ( $(parents[j]).is(filter_string) ) { return true; } } return false; } });})(jQuery);  The usage is:$(element).isChildOf(filter_string)

where filter_string is any of the expr values that can be passed to JQuery's is() function. You can get more information here

example:
<div id="parent2">  <div id="parent1">    <div id="child">    </div>  </div></div>

alert( $('#child').isChildOf('#parent1') ); Alerts "true": alert($('#child').isChildOf('#parent2') );

alert( $('#child').isChildOf('#notaparent') ); ## Wednesday, June 18, 2008 ### File Uploads with FUSE PHP MVC Framework I had a project today that required that PDF files be attached to job orders, and FUSE made the PDF upload a simple task. All I had to do was add an after_add() method in my controller to handle the upload, as shown below. The filename is the id of the job that was just added to the database, with a PDF extension: function after_add() { if ($this->is_postback() ) {         FUSE::Require_class('File/FileUpload');        $upload = new FileUpload();$upload->file_input_name = 'waiver_file';       $upload->destination_dir_create = true;$upload->allow_file_extension( 'pdf' );          $upload->destination_path = APP_BASE_PATH . DIRECTORY_SEPARATOR . 'files' . DIRECTORY_SEPARATOR .$this->job->id . ".pdf";          $upload->process(); parent::after_add(); }}  Note that I call parent::after_add() once I'm finished so that the regular FuseApplicationController after_add() method can take over and display a "changes saved" message to the user. ## Wednesday, May 21, 2008 ### postfix smtpd connection hangs with no banner I ran into an issue the other day where my Postfix mail server would accept an SMTP connection, but then would just hang without ever sending the initial ESMTP welcome banner. I discovered that because of high mail traffic, I had reached the maximum number of smtpd connections, so the new connection was stuck waiting. Since the server load was fairly low, I just raised the smtpd incoming connection limit. You can do this one of two ways First way: change the process limit just for smtpd in master.cf. Note the 200 in the "maxproc" field (the default is 100) smtp inet n - n - 200 smtpd  Second way: change default_process_limit in your main.cf file (the default is 100). This will affect all postfix processes, so be careful about taxing your server. default_process_limit=200 ## Tuesday, May 13, 2008 ### CakePHP vs FUSE MVC framework - PHP Admittedly, I'm a bit biased on the topic since FUSE has been one of my development projects for quite a while, but recently I've been working with CakePHP quite a bit. I enjoy working with Cake, but can't help but think of certain things that FUSE does (in my opinion) a bit better. I've compiled a list of a few items where I think the FUSE MVC Framework has a bit of an advantage on CakePHP. To be fair, I've also included a few things where FUSE falls short for the moment. Before I begin, I'd also like to add that I think Cake is a great framework, and I've certainly taken cues from my work with Cake to integrate new features into FUSE. Where FUSE Wins 1. The Query Builder object Fetching the last 10 active employees in CAKE: // Controller$where_conditions[] = 'Employee.active=1';$this->data['Employees'] =$this->Employee->findAll(              $where_conditions, null, null, null, 10 ); // View<?php foreach($this->data['Employees'] as $employee ) {?><div><?php e($employee['Employees']['first_name'] . ' ' . $employee['Employees']['last_name'] );?></div><?php } ?> The same thing in FUSE: //Controller$options['limit'] = 10;$options['where'] = 'employees.active=1';$this->active_employees = $this->employee->fetch_all($options );
//View<{ITERATOR active_employees}><div><{first_name}> <{last_name}></div><{/ITERATOR}>

The query builder is simply an object that, believe it or not, allows you to build queries. Nearly every method in FUSE that fetches data from the database (including aggregate methods like count_all()) can take a query builder object as one of its $options parameters, and it will apply the customization in that object to the query being executed. Above is an example of the shorthand syntax for customizing queries. For more complex customization, you can use the SQLQuery object directly, as discussed in the FUSE Wiki 2. FUSE has a real templating system Iterating through employees in CakePHP <table><?phpforeach($this->data['Results'] as $result ) {?><tr><td><?php e($result['Employees']['name']);?></td></tr><?php}?></table>

The same thing in FUSE:
<table><{ITERATOR employees}><tr><td><{name}></td></tr><{/ITERATOR}></table>

Cake's templating system is too reminiscent of inline PHP, and is sure to send your graphics guy running in the other direction. The FUSE templating system was designed to be extremely simple but incredibly robust, not to mention made to integrate as seamlessly into HTML as possible.

3. Helper methods are easier in FUSE

To generate a dropdown box of all users by first name in Cake:
//Controller$this->set('users',$this->User->generateList(null, null, null,'{n}.User.id', '{n}.User.first_name'));
// View:<?php echo $form->input('SomeModel.user_id', array('class'=>'txt', 'type' => 'select')); ?> The same thing in FUSE: <{HTML/FormHelper::Select_all( 'User', 'id', 'first_name' )}> Other compelling reasons to use FUSE over Cake include: • FUSE's User authentication and management system is much more intuitive than Cake's ACL implementation. • FUSE is also capable of doing nested included tables when fetching data. For example, using the 'include' option to any fetch method, one can account for the scenario where model A is linked to model B, and model B is linked to model C, but you want to select data from A, B, and C. Currently, in CakePHP, you would have to write the query manually. FUSE allows you to do: $options['include'] = array( 'modelA', 'modelB', 'modelB.modelC' );$iterator =$this->fetch_all( $options ); Where CakePHP Wins 1. CakePHP has a larger user base Although FUSE has been in development for several years, it was only released to the general public in January of 2008. As a result, its user base is still growing, and CakePHP is a more established framework with more documentation, forums, and third party code dedicated to it. The features of each framework are generally comparable, but the size of the CakePHP community is a huge advantage over FUSE. 2. CakePHP supports database other than mySQL FUSE 1.1 only has a database object for mySQL, but MSSQL, PostgreSQL and SQLite classes are in the works. Additionally, FUSE will not support using data from multiple databases in one project until 1.1.1. 3. CakePHP has more direct AJAX integration FUSE supports a variety of AJAX options and makes it very easy to use JQuery, but as of version 1.1, it is not as well integrated as some of Cake's AJAX functionality. 4. Your comment here I know I'm going to get flamed for leaving the "Where Cake wins" section so short, so I figured I'd offer the opportunity for readers to add their own favorite things about CakePHP by commenting on this entry ## Thursday, March 27, 2008 ### VB.net - reading extended ascii text file - StreamReader changes extended ascii to question marks Ran into an issue today reading a text file in Visual Basic .net. The file had extended ascii characters (special "curly quotes" to be exact) that were being read in as question marks because they were outside of the normal ascii character set. The solution was to set the text encoding to "default" in the StreamReader constructor, which is apparently the setting used for "Windows ANSI" text encoding: Dim fileReader As System.IO.StreamReader Dim filePath as String filePath = "C:\path\to\file.txt" fileReader = New System.IO.StreamReader(filePath, System.Text.Encoding.Default) While fileReader.Peek <> -1 curLine = fileReader.ReadLine ' do something with curLine End While fileReader.close() ## Wednesday, March 19, 2008 ### A simple Javascript slideshow I built this tool for a site I was working on, then realized it might be helpful to other people, so I thought I'd share it. The code below allows for a simple fading javascript slideshow that can be easily made by dynamic by your PHP or ASP (etc...) scripts. Included in the zip file is a working example slideshow along with the .js files you'll need to use it on your own site. You can view the slideshow in action at this link: http://www.phpfuse.net/JS_Slideshow/example/example.html You can download the code and example at the link below. Enjoy! http://www.phpfuse.net/JS_Slideshow/Simple_Javascript_Slideshow.zip ## Monday, February 18, 2008 ### MVC and you: Partners in Freedom! (Why developers should be using Model View Controller architecture) “Model/View/Controller”. To some people, it seems to be nothing but a buzzword – an ambiguous term tossed around development forums and blogs; an idea that’s vague and confusing to anyone who hasn’t actually delved into MVC development. To others, MVC is an obvious way of organizing application structure, and there’s nothing new or exciting about it. And finally there are those of us who stumbled into MVC enlightenment almost by accident and realized, wide-eyed and drop-jawed, that it’s just the “right” way to build applications. Consider me part of the third group. It wasn’t all that long ago that a Ruby on Rails-enthusiast friend of mine insisted that I start moving toward MVC development for the multitude of PHP projects I’m involved in. He knew I was working on a new PHP framework, and, thankfully, he was persistent in his endeavor to get me to delve into MVC sooner rather than later. My framework had the DB abstraction layer. It had the template system and the form validator and the photo handler and the AJAX module. But what was really missing was a way to tie it all together, and once I got about 20 minutes into MVC, I felt disappointed in myself for not having seen the light earlier. It just makes sense to develop within the MVC structure, and in this article I’m going to first give an introduction to what that structure is, then touch on a few reasons you should be using it, and finally describe a few hurdles you can expect to have to overcome while getting accustomed to it. What is Model / View / Controller architecture ? Model/View/Controller, or MVC, is essentially just a way of structuring application code. The idea is not new (it’s been around since 1979 according to Wikipedia), but it’s only happened fairly recently that MVC started to gain widespread acceptance as a viable way to develop web-based applications. A lot of the charge toward MVC-based web development was led by Rails, the Ruby framework that has gained significant momentum in the past few years. MVC-based web development is based primarily on the “M”, the “V”, and the “C” themselves – the data Models, the presentation (or View), and the interface between them – the real workhorse - the Controller. Each component has a specific role, which I’ve outlined below: 1. The data Model is used to store and retrieve information from the database. In most MVC frameworks, a data model class will exist for each of your database tables. Because the model extends a parent class that has very robust, often clever functionality, you can often gain application-level access to your data (e.g.$employee->first_name) with only a few lines of code. Data models can also be related to one another just as their underlying tables are related, which provides even greater functionality when accessing data across tables (e.g. $company->employees->fetch_all()). This extensible, customizable method for mapping tables to objects allows you to access your database without necessarily having to write SQL. Basic read and write operations are performed through simple class members and methods, and even more complicated queries can be done without having to manually write SQL, though that option is always available. 2. The “View” in Mode/View/Controller is the presentation layer that contains the content the end user will see. In web applications, the view will contain your actual HTML tags. Using the features of the templating system included within the application framework, you can add dynamic functionality to the HTML output while still keeping the application code and the presentation separate. Such a separation allows for cleaner code in both the scripting language and the HTML, and it allows designers and programmers to have more independent control over their respective parts of the application development. 3. The Controller is, as mentioned above, the workhorse of the application. The code in the controller is what’s actually run when a page is loaded, and it handles the “pull and push” – taking data from the database via the data models, passing that data to the view, then rendering the output. In most web-based MVC frameworks, specific URLs are attached to specific controller and methods through a routing file, so, for instance, “/Blog/View/123” might call the view() method of a BlogController object and pass it the id of 123. Sounds pretty complicated. Why switch from my tried and true methods of inline scripting? One of the greatest pitfalls a developer can succumb to is using only the language, tools, or methods he or she already knows in order to accomplish a task, rather than seeking out and utilizing the best tools for a particular project. A little bit of time spent overcoming the learning curve can often introduce efficiencies, expand your skillset, and actually save a significant amount of time in the long run. Many web developers have fallen into the trap of writing what can really be described as a collection of scripts, rather than developing a cohesive application, because the former is the way most of us were introduced to web development. The most common architecture used in, for instance, PHP development, doesn’t really qualify as an architecture at all. Most commonly, PHP-driven sites are comprised of a handful of individual script files that each handle incoming data from web forms or display data pulled from a database, but together lack any kind of standard structure. While this method is quick and dirty and works well for smaller applications, it simply does not lend itself to creating scalable, modular solutions that can be easily maintained and updated even as the size of the application grows. MVC provides a very structured but very versatile approach to web development, lending itself to being immediately scalable while providing simple interfaces to common functionality. Consider how often your application needs to perform the same types of operations on different data – often this is referred to as CRUD: Create, Read, Update, Delete. Entire websites and web-based applications can often be accurately described as offering only these four features, operating on different data. Without a framework to provide CRUD functionality, the developer has to hand write his INSERT, SELECT, UPDATE, and DELETE queries manually, not to mention coding tedious input validation methods to ensure security and data integrity. However, any MVC framework worth its salt will provide all of that functionality in just a few lines of code. Interested yet? Ok, you’ve convinced me, but what kind of learning curve can I expect? The first thing you should know about MVC development is that everything is done through objects, so a solid understanding of object orientation will be extremely helpful in moving forward with any MVC framework. Additionally, HTML output is handled via templates (views) – you will almost never see HTML and PHP in the same file when doing MVC development. Instead, data is passed to the template (sometimes automagically, sometimes manually) and is accessed through special syntax (e.g. <{myvar}>) in the template itself. The separation of application code and presentation sometimes throws people off initially, but the benefits of this method become apparent very quickly. Finally, with MVC, you will not see your .php files in the URI the way you’re used to seeing them. Instead, you define a route that points a URI to a specific method in a specific controller. For instance, mysite.com/BlogEntries/List might call the show_list() method of your BlogEntryController. Not only does this kind of URI routing offer almost limitless customization as to what your URI will look like, it can also be used as an SEO (Search Engine Optimization) tool. Which framework do you recommend? Personally, I use FUSE (http://www.phpfuse.net). However, that’s a bit of a shameless plug, since FUSE is my own framework. (See the video below for how you can build a database-driven web app in about 5 minutes :-) Having worked with other frameworks, I honestly believe that FUSE is the easiest MVC framework for PHP, especially as far as initial entry is concerned. There are a slew of other great frameworks available, however, including Cake and Symfony for PHP, Rails for Ruby, Grails for Java, and many, many more. ## Friday, February 01, 2008 ### subversion "working copy is corrupt" when trying to commit I was having an issue today with a new SVN repository when trying to commit a bunch of files and folders I had copied in to the now-versioned project directory. Most of the files would commit, but a few would fail with "working copy is corrupt". I discovered while browsing the project directory (outside of my IDE, which hides .svn folders) that a few leftover ".svn" folders existed because some of the folders I copied in had come from another versioned project directory. I deleted the .svn folders, re-committed, and everything was ok! ## Friday, January 25, 2008 ### Five Things Web Developers Should Stop Doing This may not come as a surprise, but I spend a lot of time on the Internet. Whether it’s browsing around for my own enjoyment or diligently working on a web-based application, I end up seeing both the end result and the inner workings of a lot of other peoples’ development work. And while a large majority of design elements are ultimately a matter of preference, there are certain web development techniques and implementation choices that I find myself shaking my head at, and I’d like to address a few of them here. The following is a list of five things that, in my opinion, web developers should simply stop doing. 1 – Including application code and HTML in the same file Although many web scripting languages are tailored for alternating between application code and HTML by use of special tags, the failings of this architecture become apparent fairly quickly when developing robust web applications. Not only does this inline scripting method create messy, oftentimes confusing code, but it can discourage effective use of functions and introduce difficulty when delegating the roles of designer and programmer to different people who may not share one another’s skill sets. The answer here is to use a templating system to separate the application code from the HTML presentation. Templating functionality is widely available for any web development language, and is an integral part of pretty much any development framework (e.g. Ruby on Rails, CakePHP, FUSE). 2 – Embedding video with a technology other than Flash Full Motion Video Until Flash FMV became widely available, a common method for video playback on websites involved encoding multiple versions of the same video, then asking the user which player he or she preferred to use (e.g. RealPlayer, Windows Media, or Quicktime). This was always a necessary evil, as developers needed to ensure that the site content was available to all visitors. However, presenting potentially confusing video preference questions to the user can often lead to abandonment, not to mention that encoding, uploading, and linking multiple versions of the same video can be a time consuming process. Thanks to the introduction of full motion video capabilities in Adobe Flash, which has shipped alongside the most popular browsers for several years, developers now have some level of certainty that at least one video player will be available to the majority of users. Additionally, Flash FMV prevents the need to spawn an external application for playback, which is another scenario that can lead to abandonment if the site visitor is unsure of how to answer their browser’s security questions. Although sites such as YouTube have unfortunately given many people the impression that Flash FMV is only capable of low quality videos with poorly synced audio, this is simply not the case. Adobe has even launched a “Flash HD gallery” (available at http://www.adobe.com/products/hdvideo/hdgallery/ ) that showcases Flash’s HD playback abilities. However, most embedded videos (news clips, etc) are short, small clips that download quickly, so even a medium or low quality encode will suffice. If your specific needs dictate that you must leverage the more advanced features of players like Quicktime or Windows Media, then you will have to use what best suits your end goal, but otherwise, stick with Flash. 3 – Implementing Flash pieces that introduce custom UI elements Flash is a phenomenal addition to any developer’s toolkit, and well designed Flash pieces can significantly enhance both the aesthetics and functionality of a website or web-based application. However, one thing that many Flash developers fail to steer clear of is overusing Flash where it’s not necessary, to the point of introducing custom user interface elements that can end up hampering usability. As an example, consider something that’s unfortunately fairly common – a Flash-based block of text with a scrollbar that is also implemented within the Flash piece itself. Not only is this an unnecessary use of Flash, since the same effect can be accomplished with fairly simple CSS, but you may be alienating visitors who simply aren’t tech savvy enough to adjust their understanding of the browser’s UI elements on the fly. It may not be apparent to some users that they’re even looking at a scrollbar, especially if the bar is stylized or implemented in such a way that it doesn’t behave like the standard scrollbar. Your visitor is used to the way their browser functions and how they use its features to browse the web, so your best bet is not to alter basic UI elements. 4 – Using long query strings where they’re not necessary Most web applications rely on the URI’s query string to bring in relevant data that is acted upon by the application code, but poor design choices often cause the query string to grow to unreasonable, unnecessary lengths. Long query strings can severely hamper the ease with which users can link to particular pages on your site, so you could very well be losing visitors because they didn’t quite get the full query string when a friend copy & pasted it over to them. The first thing to do to clean up your query strings is simply to use small identifiers for both variables and values. Try to use numeric IDs instead of long text strings to identify a specific resource, and keep your variable names short. You should also avoid passing data that could easily be extracted from data you already have. For instance, don’t pass both an item name and item id through the query string – you can just pass the item id and pull the name from the database. If you want to go a bit further in cleaning up or even eliminating query strings, look into URI rewriting. URI rewriting is a fairly simple process by which the friendly URI the user sees (e.g. /Blog/2008/01) is transparently translated into something more useful on the server side (e.g. blog_list.php?year=2008&month=01). Nearly all Model-View-Controller frameworks (Ruby on Rails, FUSE, CakePHP, etc.) have advanced techniques for rewriting the URI on the fly. 5 – Sizing images by means of the width and height attributes of the img tag This one should be a no-brainer, but I still see it fairly regularly. While it’s quick and easy to force an image down to certain size by using the tag, you’re doing yourself a disservice in at least two ways by utilizing this technique. The first problem with this method of image sizing is that web browsers aren’t particularly good at shrinking or enlarging images. The browser doesn’t do any kind of resampling, so you often get a pixelated version of your original image, even when shrinking it. The second issue with sizing images by way of the browser is that you may be wasting a lot of bandwidth. An image that’s 1000 pixels by 800 pixels has a much larger filesize than one that’s 200 by 160 pixels, so if you’re forcing it to appear at the smaller size anyway, you’d be transferring a lot of extra data for no reason by resizing it on the client side. To resize images to the size you need, use any of the widely available free tools or websites that allow you to do so. So there you have it – just a few things I’ve seen during the course of my Web travels that I personally think should be done away with. Especially as development trends continue to shift toward more user-friendly, AJAX-enabled “Web 2.0” applications, it’s important to remember to leave behind techniques that, although familiar, have either been deprecated or were never great ideas in the first place. ### PHP Security in a shared hosting environment Since its inception in 1994 as a set of basic development components, PHP has grown into one of the web’s most powerful development engines, having since been installed on literally millions of servers worldwide. And although PHP offers both the versatility and the built-in functionality to run in a reasonably secure fashion, most of those servers are configured in such a way that PHP scripts are at high risk for compromise. Most PHP-enabled webservers are configured in such a way that the mod_php Apache module is loaded along with Apache itself, thereby allowing HTTP requests to be passed through the PHP engine, which preprocesses the data before it is sent to the client. While this configuration provides a simple, efficient way to get PHP up and running, it raises security issues when working in the most common webserver environment: shared virtual hosting. Generally, it is unnecessary and wasteful to dedicate an entire server to hosting just one website. Since most sites demand only a small fraction of a server’s available resources, it is more common to have one server be home to a large number of virtual hosts. A virtual host is simply a configuration entry that points requests for a specific URL (for instance, www.google.com) to a particular directory (for instance, /home/www/mydomain.com). The shared hosting model, though economical, immediately presents a security concern, since the HTTP server (for instance, Apache or Microsoft IIS) needs to have a considerable amount of control over the files and directories that are to be served to the client. If your application offers the ability to upload files posted through web forms, the problem is further compounded since the HTTP server now needs write permission on the destination directory. In the common virtual hosting configuration discussed above, if the HTTP server has write permission to that directory, then any user running a PHP script on that same server can also write to the directory. Obviously, this presents a major security concern. However, there are steps that can be implemented, as a server administrator or as a user, that will eliminate or mitigate the security issues, or at least isolate individual users so that a script exploit on one host cannot easily affect other hosts on the same server. In this article, I will discuss a few methods for more securely configuring PHP, and will offer some security-conscious techniques to use when writing applications. For the sake of convenience, I have grouped the article into three categories: Configuration directives and environment settings that can only be changed by a server administrator, a basic overview of PHP wrapping for the application developer, and general practices for securing PHP code. Even if you are not administrating your own server, I recommend reading through the first section in order to gain an understanding of the problem so that you can know what to expect from your web host. This article makes the assumption that your environment has PHP running on a Linux/Unix variant, with Apache acting as the HTTP server. From the administrator’s perspective: Configuring your shared PHP environment As the administrator of a virtual hosting server, you have full control over the HTTP server and the PHP engine, which is the ideal condition for tuning and securing your environment. First, let’s talk about separating the PHP interpreter from the Apache server, so that we negate the file permissions problem discussed above. The PHP interpreter can be invoked in three different ways: as an Apache module (discussed above), as a CGI binary, and as a CLI. Since the CLI (Command Line Interface) isn’t relevant for serving web pages, I won’t be addressing it in this article. As mentioned above, the most common (and often default) method for invoking PHP is as an Apache module. However, let’s look at an alternative way of invoking PHP – namely, as a CGI binary. When invoked as a CGI binary, Apache loads the PHP interpreter only when needed, passing necessary input (environment variables, POST data, etc) to the PHP executable, then collecting the output and sending it to the client. In this scenario, the PHP process is separated from the Apache thread, which makes it possible to run the processes as different users, thereby eliminating the permissions problem discussed above. However, by default, PHP is run as the Apache user, so we haven’t yet solved the problem simply by running PHP as a CGI binary. Our next step is to “wrap” PHP so that it is invoked as a user that we specify, not as the Apache user. Note: Installing PHP as a CGI Binary can introduce other security concerns that are worth being aware of. While PHP is generally secure out of the box, It is advisable to take a look at http://us.php.net/manual/en/security.cgi-bin.php. While Apache does have its own mechanism, suEXEC, for wrapping CGI programs, I will not be discussing it in this article. Instead, we’re going to look at another open source package: suPHP. Written by Sebastian Marsching, suPHP is a fairly simple Apache module that nicely wraps the PHP binary “in order to change the UID of the process executing the PHP interpreter” (suphp.org). In my experience, installing suPHP has always been a fairly pleasant (as far as these things go) endeavor. You will need to refer to the suPHP instructions at http://www.suphp.org for installation information for your particular UNIX distribution, but as a lightweight application that makes use of Apache’s dynamic module API, installation of suPHP should be trivial. There are many articles on using PHP’s ini directives to help lockdown your server, and although that is not the focus of this article, I would now like to briefly touch on a few directives you should be aware of. First, be sure to enable PHP’s safe_mode. Although some older applications have trouble running with safe_mode enabled, most have been updated to account for this directive, and its benefits are simply too numerous to ignore, especially if you are not able to use a PHP wrapper such as suPHP. Safe mode will be removed in PHP 6 in favor of alternative methods of implementing file and directory security, but for now, you should leave it enabled. Next, if you cannot implement suPHP or another PHP wrapper, it’s a good idea to set open_basedir in all of your virtual hosts. Set “php_admin_value open_basedir /path/to/vhost/root” as a directive in your virtual host configuration to ensure that PHP is restricted from reading any files outside of the virtual host’s document root. Finally, have a look at the disable_functions directive. While you do want to make sure that your security procedures don’t prevent your users’ applications from running as they should, it’s often the case that few, if any, users will need any of the more potentially hazardous functions such as passthru(), exec(), and shell_exec(). If it is the case that none of your users need these functions, it’s a good idea to disable them. (Note: In the event that only one user or application needs these functions, suPHP allows you to specify individual php.ini files for specific virtual hosts, which offers a middle ground between allowing these functions globally and restricting applications that need to use them for legitimate purposes.) From the developer’s perspective: Why do I want a PHP wrapper? Why do you want suPHP, or a PHP wrapper at all? Let’s look at a very common example – uploading images. It’s a common condition that an application needs to accept image uploads via the web, and many developers operating in a shared hosting environment have run into the problem where, on the first try, PHP displays a “permission denied” error when trying to move the uploaded file into its destination directory. Our User vs. HTTP server permission problem is back again, where the HTTP server - user “www” or “apache” -does not have the proper permissions to write to a directory owned by the developer’s account. The common solution to this problem is to set the permissions on the destination directory to 777, giving all users system-wide read, write, and execute access. While this does work and your uploads can now flow freely, you’ve just ensured that any other user on the system – there are probably hundreds – could very easily issue an “rm –rf /path/to/your/uploads”, which would quickly and effectively delete everything in the directory. While I personally like to think that there exists camaraderie between users on the same server, this probably isn’t true, and you also have to consider that someone else’s account may have been compromised (probably by a lack of input checking on an upload form – more on that below). With suPHP (or another PHP wrapper) enabled, you are free to leave your upload destination directories with the same permission as your other web-accessible files – namely that only your user account has write access, and the Apache user can read files and traverse directories. In fact, if your user account is in the same group as the Apache user, you can set these directories to have permission of 750, which is much more restrictive than a wide-open 777. The primary downside to using a PHP wrapper is that there is in fact a performance hit, since the PHP interpreter has to be invoked for every request, rather than being started as part of the Apache server. However, in my experience, the performance decrease is generally unnoticeable. If your application is extremely performance-critical, you will want to run benchmarks before deciding to use a wrapped PHP environment, or consider graduating to your own dedicated server where you can ensure that only you and your developers have any kind of access to your application files. In the dedicated server scenario, the security concerns of using PHP as an Apache module are largely mitigated. ( Note, however, that if one site on your dedicated server is exploited in a mod_php setup, other sites or files will most likely be vulnerable as well, whereas a server running suPHP with PHP’s safe_mode enabled and open_basedir directive configured will generally be able to jail the attack to one virtual host’s document root) General practices for PHP application security At this point, I’d like to briefly go over just a few coding techniques you can use to increase the security of your application. Please understand that this is by no means a comprehensive list, and simply adhering to the suggestions below does not ensure that your application is secure. However, you should make it a point to be security conscious when writing code, rather than trusting your environment to eliminate or mitigate any potential attacks on your application. 1. Sanity-check your data - This is probably the simplest and most effective way of preventing exploits in your application. Sanity checking just means that if you’re expecting the user to enter a number, make sure you actually received a number. If you’re expecting a string with alphanumeric characters only, verify that that’s what you got. Also, never trust this kind of validation to javascript only, as javascript can easily be disabled on the client side. Finally, never pass user data directly to an SQL query without validating it first. While PHP’s magic_quotes mechanism is great for helping to prevent SQL injection attacks (an attack where a user can enter data in such a way as to run their own arbitrary queries), again you should not rely exclusively on the environment for your application security. 2. Check the type and extension of uploaded files. - Allowing file uploads is inherently risky, but very often it’s a necessary part of an application. PHP allows you to gain a lot of information about uploaded files before they’re ever written to their final destination, so make use of the information contained in the$_FILES array to ensure that you’re getting the type of file you’re expecting. A basic way of validating the file type is simply to ensure that the extension of the file indicates that it is (or is purported to be) the type of file you’re expecting. A common exploit for upload scripts is for an attacker to upload a malicious PHP script to your site, then browse to the uploaded script to gain control of your files. Even a basic check to make sure that, for instance, only files with a .jpg extension are allowed to be uploaded would prevent this type of exploit. However, I also recommend verifying the MIME type of the file, which is contained in the \$_FILES array under the key ‘type’, and will look like: “image/jpeg” or “application/pdf”. Be as restrictive as possible – rather than validating against a list of extensions that are NOT allowed (php, exe, etc), check to make sure that the extension and/or MIME type matches a small group of file types that ARE allowed.

3. Use a .php extension for ALL files with PHP code contained in them.

Often I come across files in PHP projects that have a .inc extension, because they are meant to be included, not browsed to directly from the web. This is a common condition, but there’s a potential security issue here if those files contain any sensitive data (e.g. database passwords). Because .inc files are not parsed by the PHP interpreter, they can be passed directly to the client side if they’re available via a web request, which would allow anyone to read the php code directly. Hopefully the directory these files reside in is denied read access by webserver rules (see below), but even so, there’s no sense in risking an accident where the user ends up being able to browse directly to the file. Use .inc.php.

4. Make use of your webserver’s access control rules (e.g. .htaccess)

- Even if all the php files in your application have a .php extension as discussed in #3, you should still make use of your HTTP server’s access control to prohibit any files in sensitive directories from being served via the web. For instance, if you keep your database passwords in “include/db.inc.php”, this file, and all files in the include/ directory should be prevented from being served via the web. Even though the .php extension will ensure that client-side users can’t read the code if the PHP interpreter is functioning, there is the potential condition that the HTTP server has loaded without the PHP interpreter. Botched upgrades or configuration errors can sometimes cause this condition, and in the event that someone browses to a PHP file without the PHP interpreter ever having been loaded, they will again see the code just as you do when editing the files. In Apache, disabling directory access is usually as simple as creating a file called .htaccess (note the leading period) in the directory, then adding the line: “Deny from All” (no quotes) to that file and saving it.