Security can be easy

It’s often the case that security is an inconvenience and gets in the way of usability and ease of use. There are exceptions though and for a number of weeks I’ve been playing with the Yubikey (thanks to Schack) from Yubico.

It’s a small device, which plugs into a USB port, and to the computer acts as a keyboard. It has some advanced security build-in with the ability to generate one-time verifiable passwords, but is incredible easy to use – plug it into the USB port and press the single button when you need to sign in to services supporting the Yubikey.

I’ve been useing it with OpenID sign-ins and WordPress logins, and it’s worked flawless everytime. The only important thing is to remember the Yubikey – without the little hardware token, you’re lost. If you need a secure sign-in solution, listen to security now to get the technology explained and contact Yubico, it’s seems to be as affordable as it is easy to use (and no, the post isn’t paid or sponsored, nor do I know anyone at Yubico).

Tips for developing WebApplications

So it been awhile since I sort of promised some tips on developing web applications fast. This post contains my five generic tips, which may apply to you (but then again – your mileage may vary). I’ve tried to abstract the advice and not stick to PHP development here, though it’s by far the world I know best.

1. Spend time on the data model

Too often a data model is driven by code, not careful thinking, and this often causes problems has the web application develops and expand over time. When developing a site, which needs data stored in a database (as most web applications tend to do), do try to spend some time on the data model – which tables do you need, what fields should the tables have, are the naming logical and consistent, what constraints and limitations does the data model offer.

It’s often far easier to refactor code and functionality than it is changing the database tables and fields, and it can be a tremendous help while maintaining an application if you clearly understands the data its working with.

2. Get the basics working first and fast

It often takes a while to get a complete web application finished. Help motivating yourself by getting the core working first and then add features and frosting once it’s working. By having a working skeleton, I usually find it much easier guess how much work needs to finish the various features than I do guessing the work needed to complete a section (ie. admin interface) on the application.

3. Think performance and security from the beginning

Remember to think of performance and security from the very beginning. Sometimes performance – or security – may cause you to layout the data model in a certain way or structure the application a certain way. While you may be able to add performance and security during refactoring, it’s often much harder than it is keeping both in mind from the very beginning.

4. Don’t fix the future; focus on the current needs

I don’t care about the next version. I really don’t want an infrastructure in place, which can help me build the future versions of the application. I want the features and infrastructure in place for the current version I’m working on – and frankly the features in the next version has a tendency to change before I’m ready to develop them.
Too often future-proofing an application clutters the code and too often the clutter doesn’t ever come to any good use – it’s just noise once we get to the real future features of the application.

5. Naming matters

Do think of what names you use on functions, classes, variables and database tables through out the application – and choose something meaningful. A variable named $x doesn’t hint what it’s used for. Help yourself (and other looking at the code) by hinting the uses of various things by choosing meaningful names.

Your mileage may vary

It’s always fun to read articles with tips and tricks by other developers and see how they figure “best practices” are handled. Most developers do seem to thing they observations and practices are easily adopted by anyone, and should be accepted without any argumentation or reasoning behind the advice.

One of the nice examples of this, was a story called “5 tips and tools to develop php applications fast”, and while it may apply to the web applications developed by the author, it’s one of those where I question who is supposed to be the audience. All 5 tips are often found in many textbooks and other stories on the net, and in my experience they are all wrong (more or less naturally).

1) Use a MVC framework – it’s an industry accepted standard.
MVC as an idea may be accepted, but there are many ways of implementing the MVC – even with the various PHP frameworks. If you know the MVC pattern it may be an idea to look at the frameworks supporting it, but don’t get carried away.

  • If all your old code is non-MVC code and you don’t have any plans to MVC-refactor it, it may cause you maintenance overhead to try a completely different style  on a single project/web application.
  • If you aren’t used to using a (MVC) framework you need to invest time in learning it.

Do keep an eye on the overhead caused by the framework. If you web application has a heavy load and the framework causes 5-10% overhead, it may a bad idea.

2) Ajax Frameworks
Great idea.. but to utilize the power of an ajax framework, you need to learn how to use it. Also Ajax isn’t the holy grail of the internet. Sometimes it’s cleaver to use it, other times it isn’t. By using plain old HTML and reloading the page from time to time, it may be easier to debug the application.

Do also notice, that writing great/professional quality Javascript often requires just as great skills as writing professional PHP.

3) IDEs
An IDE – Integrated Development Environment – can be a great idea if you know your way around it, and use it regularly. An IDE is – like any other tool – something you need to invest time and resources into, if you want something back – and especially if we’re talking a beast like Eclipse-based tools.

4) Database creation/management software
Let me skip comments on database creating management tools. I’ve never really used them and don’t know if they’re a pain or a benefit. I have often seen though, that too easy access to the database, leads to sloppy database design (or at least something, that isn’t thought through).

So far it’s been my experience, that people who knows the command-line clients to databases, often has invested much more time in thinking their data models through and often spend a little more time considering the table layout, the field definitions, indexes and other important aspects, than those who use a nice graphical interface.

5) Object Relational Mapping
Object-Oriented programming isn’t a holy grail, and most people doing OO programming (in my experience) doesn’t genuinely do object-oriented programming – instead they use the OO-facilities in the language to slap functions together in a class.

There’s nothing wrong with procedural programming as such, and trying to force object-orientation into your code – and even abstracting it away through Object-relational mapping, has often lead to slow performance and odd database designs. If you know what you’re doing great, but there’s no free lunch, and trying to utilze ORM without having a pretty solid grip on what’s happening and why, rarely does anything good to your productivity.

I’m quite sure for some developers, using some sort of object-relational mapping might be a gain in programmer productivity, but do keep in mind, that if you’re developing a site with massive traffic, the programmer gains may easily be crushed in added server resource requirements – spending a few minutes extra optimizing and tuning applications, may in some cases save hours and hours of cpu cycles once the solution is deployed.

Getting back to the article, I’m sure the advise is well-meant and thought through by the author, but do always – as with all advise – think it through and see how may apply to your adoptation. In may daily development some of it is quite terrible (as I often work on a site with thousands if not millions of pageviews every week).

I’m wondering what my 5 tips would be for developing PHP applications fast would be. Check back in a few days, and there may be an answer – which may or may not – apply to you.

Code archeology

I’ve been spending quite some time the past days digging around in old code (3+ years old with no or very little changes). It’s quite fun noticing what efforts pay off and which doesn’t when you go back to make chance to old stuff. I’m sure my observations aren’t generally applicable. We’re web developers and are keeping web platform (several pretty big websites on a common codebase) alive year after year.

Proper and sane variable- and function names pays off instantly. Incredibly short or misguiding function and variable names doesn’t. While it may be fun naming functions “doMagic” or likewise, chances that you – or someone else maintaining the code years after will probably not appreciate it at all.

Reasonable commenting on functions and tricky code parts often come to good use, but too much commenting is distracting. Some is good, and less is often more valuable.

Version control logs can be quite valuable, if they’re descriptive for the change made. Too often though, was the comment “misc. bug fixing”, “problem solved” or something which years later was of no use or value at all.

Documentation kept apart from the source code seem to have none or very little value. Not once have I bothered to look in the wiki-documents or any other documentation which was in the code. In the few instances I did, minor changes and bug fixes had changed the code to such an extend (while not updating the documentation) that it was misleading at best.

The only cases where the documentation apart seem to make a positive is database diagrams and descriptions. They do seem to have value.

PHP Dynamic Caching with ZendPlatform

I recently had a little fun playing with the dynamic cache available in the Zend Platform. The Zend Platform is a commercial product, which provides a number of cool professional features to a PHP setup – one of these is the option to do dynamic caching.

With dynamic caching you can cache the output of a function for a determined period and instead of doing database queries or web service calls for a feature, you can cache the results, save resources and get faster pages.

If you have access to a PHP setup with Zend Platform, but haven’t looked at dynamic page caching, here’s a little example to get you inspired and started.

Dynamic Caching Example

Let’s suppose you have a busy news website and publish lists of news from categories – on the front page, along side news articles and several other places. to retrieve the most recent articles for each category, we’ve created a class with a function to retrieve the five most recent stories from a category specified by a parameter.

Our first version may look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class NewsCategory
{
        public function getCurrent($category)
        {
                $newsLines = Array();
		if (intval($category)) {
                $sql = sprintf('SELECT * FROM news WHERE category="%n"
                                     ORDER BY date LIMIT 5', $category);
                /* Asume connection to dbserver exists and is valid */
                $res  = mysql_query('newsdb', $sql);
                $data = mysql_fetch_array($res);
                $newsLines = Array();
                while ($newsItem = mysql_fetch_array($res)) {
                        array_push($newsLines, $newsItem);
                }
        }
        return $newsLines;
        }
}

Since news doesn’t happen too often and the lists of news are displayed on many pages, we want to cache the results for an hour – and we want to make sure, that the site works, even if we for some reason decide to disable the Zend Platform.

Rewriting the class to utilize the caching API takes a few changes and the Dynamic Cache-enabled version looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
/*
 * SAMPLE CODE; NOT FOR PRODUCTION USE
 */
class NewsCategory
{
        const CACHE_TIME = 3600; // caching time in seconds. Global caching set to 1 hour.
        const CACHE_KEY  = 'NewsCategory_getCurrent_'; // prefix for caching key
 
        /**
         * Returns array with current newsitems from the specified category.
         *
         * @param integer $category
         * @return array with news items
         */
        public function getCurrent($category)
        {
                if (function_exists('output_cache_fetch')) { // caching available?
                        return unserialize(output_cache_fetch(
                                self::CACHE_KEY . $category,
                                "serialize(self::fetchCurrent($category))",
                                self::CACHE_TIME));
                } else { // No ZendPlatform Available
                        return self::fetchCurrent($category);
                }
        }
 
        /**
         * Internal function for looking up the current news identified by $category.
         *
         * @param integer $category
         * @return array - returns empty array if no news is found.
         */
        protected function fetchCurrent($category)
        {
                $newsLines = Array();
		if (intval($category)) {
                $sql = sprintf('SELECT * FROM news WHERE category="%n"
                                     ORDER BY date LIMIT 5', $category);
                /* Asume connection to dbserver exists and is valid */
                $res  = mysql_query('newsdb', $sql);
                $data = mysql_fetch_array($res);
                $newsLines = Array();
                while ($newsItem = mysql_fetch_array($res)) {
                        array_push($newsLines, $newsItem);
                }
        }
        return $newsLines;
        }
}

The changes are:

  • The public function from the first version is replaced by a method, that handles the caching. The function doing the work is now hidden in a protected method in the class.
  • If the caching API isn’t available, the new function just acts a proxy passing data through.
  • The new caching functions creates caching keys from the method name and the parameter to make them predictable and avoid namespace clashes.
  • do notice that the cached results are serialized in the cache – the cache can only store string values.

Before going crazy with Dynamic Caching do consider a few things:

  • Is the data suited for caching? (or do you need the most accurate data every time)
  • What level do you expect the cache ratio to be at? (how often is the cache utilized)
  • Are other caching mechanisms in place? (ie. proxy-servers or likewise)

Loading data from a file with PHP

In programming languages common tasks should be easy, and surprisingly often I find my self loading data from some source file into a database. With PHP loading a CSV file into the database – or posting the data to an API – is quite easy.

The only trick is to know the functions file_get_contents, split (and possibly list). The routine goes something like this:

1
2
3
4
5
6
7
8
9
10
$fileName = 'rawData.csv';
 
$content = file_get_contents($fileName);
$rows = split("\n", $content );
 
foreach ($rows as $row) {
	list($item1, $item2, item3) = split(';', $row);
 
	// Do something interesting...
}

The script gets the contents of file into a variable ($contents). Split the contents into an array ($rows) and the forach loop across each row. The first line of the loop splits the row into three fictitious items and the rest of the loop could do something interesting to the data read from the file.

I usually run scripts like these from the command-line (yes, you can do that with PHP scripts).

PHP: Removing an item from an array

If you have a php array, but need to nuke an item from it, the unset function is just the tool to do that. When iterating through the array after removing the item, you should be slightly careful.

The unset function does remove the item, but it doesn’t “reindex” the array, so you should traverse the array by index-numbers after removing the item. The issue is probably best illustrated by this example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$myArray = Array('item 1', 'item 2', 'item 3');
 
// remove item
var_dump($myArray);
 
unset($myArray[1]);
var_dump($myArray);
 
// Failing bad loop
for ($loop = 0; $loop < count($myArray); $loop++) {
        echo $myArray[$loop] . "\\n";
}
 
// Good loop
foreach ($myArray as $item) {
        echo $item . "\\n";
}

You can dowload the above example code here.

A Premium look

Welcome to a fresh look on netfactory.dk. I’ve never been quite satisfied with the old look, and stumbled across the WP_Premium theme which looked quite nice. It’s now live in a lightly hacked version.

Therre are quite a few things I’d like to hack further but in the spirit of the Internet Way of doing things, it’s probably best to release something slightly broken – and let updates happen “when I find the time”(tm).

What is twitter?

One of the hottest sites on the web for more than a year is twitter, but what is Twitter? – I’ve tried a few times to explain it, and while it may be a fun task, it has often become quite a mess. This is an attempt to capture the most successful explanation of twitter.

The core of Twitter is a combination of three different characteristics:

  • Twitter is like a blog – An author publish content. It may be personal, it may be themed, it may be interessting – there are no set rules for the contents except those set by the author.
  • Twitter is like an SMS – There are a 140 character limit on each piece of content. If you need more, you need to split contents in several “twits”.
  • Twitter is a network – It’s no just a website. Through build in services and APIs you can connect with twitter through SMS, desktop clients, Instant messaging and many other ways. Besides a technical network, it’s also a social network where you can follow other interesting users, communicate with other users (private or in public).

That’s pretty much the core.

There are a lot of other features stuff you can do, and while many has public feeds, you can even choose to keep you twittering private and only allow people you authorize to see you contents on twitter.

Another interesting use of twitter is when applications start interacting with it. When I post contents on the WordPress-based blog, a twit is automatically posted announcing it and thus twitter may in some extend be an alternative to my RSS feeds. See also iWantSandy of an interesting application interaction through twitter.

If you want to follow my ramblings on Twitter go ahead, they’re public.
If you have a better definition of what twitter is, please post a comment.