Philadelphia Reflections

The musings of a physician who has served the community for over six decades

Volumes

George IV

George and Computers(1)

I got him into computers around 1960. He soon far surpassed me.

New topic 2016-12-20 00:21:28 contents

XHTML vs. HTML

The markup language used by web browsers continues to evolve. The most current version (as of April 2009) is XHTML 1.1, an XML version of HTML.

Many browsers, most particularly IE, do not support XHTML. Technically speaking, they support only the "text/html" mime type, not "application/xhtml+xml". Lots of web developers have gone to the trouble of sticking closing tags ( />) in their BR, HR, META and INPUT tags and a DOCTYPE at the top but then serve the code as "text/html".

This produces a syntactic mish mash which may be worse than using strict HTML 4.01.

Why "worse"? Because of the possibility of unintended results from providing incorrect instructions to the browser. If you care about the output produced by the browser, which most developers and content providers emphatically do, then you have to be careful about what instructions you give the browser. You simply cannot count on getting what you want if what you're telling the browser to do is syntactically incorrect.

However, it's a little difficult to see just what good XHTML is:

Internet cognoscenti speak disparagingly of "tag soup" but the Internet is a lot more about content than it is about syntax, so who really cares?

Well, somehow, I do. A little. Since we use PHP on this site, we have the opportunity to figure out what features are supported by a browser and render the correct types of tags, mime-types, etc.

Check out the HTTP headers and the page source to see the following script in action:

  1. It renders XHTML 1.1 whenever it encounters a browser that can support it
  2. It uses output buffering (which demonstrably if illogically improves rendering response time)
  3. It sends the whole thing using gzip compression if the browser will support it
  4. But also, it concedes certain issues based on experience for the sake of a smoothly-operating website
<?php
//
//  This script figures out what kind of mime type (HTML vs XHTML) the browser supports and sends the correct headers
//  It also initiates compression, specifies cache-ing and sends other <meta http-equiv headers
// 
//  My thanks to http://www.workingwith.me.uk/articles/scripting/mimetypes for the basic idea and structure
// 
//  $_SERVER["ACCEPT"] describes the mime_types a browser supports in a comma-separated list:
// 
//    mime_type,mime_type,mime_type
// 
//  If a browser prefers one mime_type or group of mime_types, it adds a q-value
// 
//    mime_type,mime_type;q=x.x, mime_type,mime_type,mime_type,...,mime_type;q=x.x
// 
//  The q-value is a number between 0.0 and 1.0 ... the higher the number, the greater the preference
//  The idea is that if we can serve more than one mime_type we should serve the browser's higher preference
//
//  ob_start("ob_gzhandler"); does all the work to compress the output if the browser can handle it
//  ob_start("fix_code"); calls the "fix_code" function instead, so initiating gzip is my responsibility
//
//  $_SERVER["HTTP_USER_AGENT"] is an opaque decription of the browser itself
//
//  $_SERVER['HTTP_ACCEPT_ENCODING'] describes compression capabilities
//
//  I output these three variables as an HTML comment so I can debug things more easily
//
//  Despite my desire to do things "right", you will see I accomodate myself to the reality of user-supplied content 
//  and browser peculiarities in order to have a working website
//

function fix_code($buffer)
  {
  #
  # Called for HTML browsers to delete all the lovely close-brackets
  # it's up to me to initiate the gzipping because ob_start is called by "fix_code" instead of "ob_gzhandler"
  #
  if (stristr($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip'))
    {
    header("Content-Encoding: gzip"); // notifies the far-end to un-gzip 
    return (gzencode(str_replace(" />", ">", $buffer),6,FORCE_GZIP));
    }
    else
      {
      return (str_replace(" />", ">", $buffer));
      }
  }

#
# default values
#
$charset          = "UTF-8";       # See http://en.wikipedia.org/wiki/UTF-8
$mime             = "text/html";   # Plain vanilla
$cache_control    = "max-age=200"; # Cache expires after 200 seconds

$xhtml_q          = 0;
$html_q           = 0;

# see http://www.w3.org/QA/2002/04/valid-dtd-list.html
$DOCTYPE_xhtml11  = "<!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.1//EN' 'http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd'>\n"; 
$DOCTYPE_xhtml10  = "<!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Strict//EN' 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd'>\n";
$DOCTYPE_wap      = "<!DOCTYPE html PUBLIC '-//WAPFORUM//DTD XHTML Mobile 1.2//EN' 'http://www.openmobilealliance.org/tech/DTD/xhtml-mobile12.dtd'>\n";
$DOCTYPE_html401  = "<!DOCTYPE html PUBLIC '-//W3C//DTD HTML 4.01//EN' 'http://www.w3.org/TR/html4/strict.dtd'>\n";
$DOCTYPE_html401l = "<!DOCTYPE html PUBLIC '-//W3C//DTD HTML 4.01 Transitional//EN' 'http://www.w3.org/TR/html4/loose.dtd'>\n";

$html_xhtml       = "<html xmlns='http://www.w3.org/1999/xhtml' xml:lang='en'>\n\n";
$html_iphone      = "<html xmlns='http://www.w3.org/1999/xhtml' xml:lang='en' manifest='iphone.manifest'>\n\n";
$html_html401     = "<html lang='en'>\n\n";
$html_html401_IE  = "<html lang='en' xmlns:v='urn:schemas-microsoft-com:vml'>\n\n";  # xmlns:v='urn:schemas-microsoft-com:vml' is recommended by Google for maps display using IE
$html_plain       = "<html>\n\n";

# parental control tag
$pics_Label       = '(pics-1.1 "http://www.icra.org/pics/vocabularyv03/" l 
	gen true for "http://philadelphia-reflections.com" r (n 0 s 0 v 0 l 0 oa 0 ob 0 oc 0 od 0 oe 0 of 0 og 0 oh 0 c 0) 
	gen true for "http://www.philadelphia-reflections.com" r (n 0 s 0 v 0 l 0 oa 0 ob 0 oc 0 od 0 oe 0 of 0 og 0 oh 0 c 0) 
	gen true for "http://search.freefind.com" r (n 0 s 0 v 0 l 0 oa 0 ob 0 oc 0 od 0 oe 0 of 0 og 0 oh 0 c 0) 
	gen true for "http://www.search.freefind.com" r (n 0 s 0 v 0 l 0 oa 0 ob 0 oc 0 od 0 oe 0 of 0 og 0 oh 0 c 0) 
	gen true for "http://statcounter.com" r (n 0 s 0! v 0 l 0 oa 0 ob 0 oc 0 od 0 oe 0 of 0 og 0 oh 0 c 0) 
	gen true for "http://www.statcounter.com" r (n 0 s 0 v 0 l 0 oa 0 ob 0 oc 0 od 0 oe 0 of 0 og 0 oh 0 c 0) 
	gen true for "http://c3.statcounter.com" r (n 0 s 0 v 0 l 0 oa 0 ob 0 oc 0 od 0 oe 0 of 0 og 0 oh 0 c 0) 
	gen true for "http://www.c3.statcounter.com" r (n 0 s 0 v 0 l 0 oa 0 ob 0 oc 0 od 0 oe 0 of 0 og 0 oh 0 c 0))';

# I include the following HTML comment for my ongoing debugging purposes
$show_info        = "<!-- \nHTTP_USER_AGENT      $_SERVER[HTTP_USER_AGENT]\nHTTP_ACCEPT_ENCODING $_SERVER[HTTP_ACCEPT_ENCODING]\nHTTP_ACCEPT          $_SERVER[HTTP_ACCEPT]\n -->\n\n";

# note that I eval $prolog_type below so that the xml header (if any) gets the right charset
$prolog_type      = '$DOCTYPE_html401l $html_plain $show_info';

#
# the logic
# 

# W3C Validator
if (stristr($_SERVER["HTTP_USER_AGENT"],"W3C_Validator")) 
  {
  ob_start("ob_gzhandler");
  $mime        = "application/xhtml+xml";
    # UTF-8 produces character-type errors
    $charset     = "iso-8859-1";
  $prolog_type = '$xml_header $DOCTYPE_xhtml11 $html_xhtml $show_info';
  }
  else
    {
    # fancy wap-enabled handheld device
    if(stristr($_SERVER["HTTP_ACCEPT"],"application/vnd.wap.xhtml+xml")) 
      { 
      ob_start("ob_gzhandler");
        # per http://www.ready.mobi/ and http://www.w3.org/TR/mobileOK-basic10-tests/ application/xhtml+xml is preferred
//      $mime        = "application/vnd.wap.xhtml+xml";
        $mime        = "application/xhtml+xml";
      $prolog_type = '$xml_header $DOCTYPE_wap $html_plain $show_info';
      }
      else
        {
        # non-wap xhtml-enabled browser
        if(stristr($_SERVER["HTTP_ACCEPT"],"application/xhtml+xml")) 
          { 
          # retrieve the q values for "application/xhtml+xml" and "text/html"

          if (preg_match('%application/xhtml\+xml[^;]*?;q=([1|0]\.[1-9]+)%i', $_SERVER["HTTP_ACCEPT"], $matches)) 
            {
            $xhtml_q = (float)$matches[1];
            }

          if (preg_match('%text/html[^;]*?;q=([1|0]\.[1-9]+)%i', $_SERVER["HTTP_ACCEPT"], $matches)) 
            {
            $html_q = (float)$matches[1];
            }

          # if the q value for HTML is greater than for XHTML
          # then treat output as HTML 4.01 strict (Opera 9.64, for instance)

          if($html_q > $xhtml_q) 
            {
            ob_start("fix_code");
            $mime        = "text/html";
              # UTF-8 produces character-type errors
              $charset     = "iso-8859-1";
            $prolog_type = '$DOCTYPE_html401 $html_html401 $show_info';
            }

            # otherwise, go with XHTML
            else
              {
              ob_start("ob_gzhandler");
                # for the time-being application/xhtml+xml is too strict for us: unless your tags are PERFECT, it blows up
//              $mime        = "application/xhtml+xml";
                $mime        = "text/html";
                # UTF-8 produces character-type errors
                $charset = "iso-8859-1";

              # see "Safari Web Content Guide for iPhone OS" for cache manifest description
              if (stristr($_SERVER["HTTP_USER_AGENT"],"iPhone")) 
                {
                $prolog_type = '$xml_header $DOCTYPE_xhtml11  $html_iphone $show_info';
                }
                else
                 {
                  $prolog_type = '$xml_header $DOCTYPE_xhtml11  $html_xhtml $show_info';
                 }
              }
            }
          
          else
            {
            # plain text/html browser
            if(stristr($_SERVER["HTTP_ACCEPT"],"text/html")) 
              { 
              ob_start("fix_code");
              $mime        = "text/html";
                # UTF-8 produces character-type errors
                $charset     = "iso-8859-1";
              $prolog_type = '$DOCTYPE_html401 $html_html401 $show_info';
              }
              else
                {
                # if the browser doesn't specify any X/HTML mime type, treat like HTML 4.01 Transitional (IE 7, for instance)
                ob_start("fix_code");
                $mime        = "text/html";
                  # UTF-8 produces character-type errors
                  $charset     = "iso-8859-1";
                $prolog_type = '$DOCTYPE_html401l $html_plain $show_info';
                # if IE then include Google's recommended "xmlns:v  ..." 
                if(stristr($_SERVER["HTTP_USER_AGENT"],"MSIE")) 
                  {
                  $prolog_type = '$DOCTYPE_html401l $html_html401_IE $show_info';
                  }
                }
            }
        }
    }

#
# output the mime type, prolog type and other <meta http-equiv= variables
#
header("Content-Type: $mime; charset=$charset");
header("Content-Language: en-us");
header("Vary: Accept");

header("Cache-Control: $cache_control");

header("Content-Script-Type: text/javascript");
header("Content-Style-Type: text/css");
header("imagetoolbar: no");

// parental controls from http://www.icra.org/
header("pics-Label: $pics_Label");

// privacy header created at http://www.p3pwiz.com/
header("P3P: policyref=\"http://www.philadelphia-reflections.com/w3c/p3p.xml\", CP=\"NID DSP NOI COR\"");

$xml_header       = "<?xml version='1.0' encoding='$charset' ?>\n";
eval("\$prolog_type = \"$prolog_type\";");

print $prolog_type;
?>

Here's an interesting article on Doctype Switching: http://gutfeldt.ch/matthias/articles/doctypeswitch.html

The Philadelphia Reflections webmaster: George IV

(my thanks to http://centricle.com/tools/html-entities/ for HTML encoding)

Webpage Printing

This site offers a Print button for all Reflections and Topics. Formatting the text on the pages to print nicely works quite well; but how to specify what to do with images remains a bit unclear (as of August 2006). Although 95% of users employ Internet Explorer because Microsoft supplies it free with new computers, IE is just about the worst browser to use for printing. Safari is much better, and Firefox is pretty good. Opera is also satisfactory, but Internet Explorer is not recommended. The other browsers are free; find them in Google and download them. For the usual user, that's all you have to know.

If you are curious about the technicalities, read on. The "trick", if it can be called that, to special print formatting is the media attribute for CSS styling. The main stylesheet for this website is called in a LINK statement as follows:

<link rel="stylesheet" type="text/css" media="all" href="stylesheets/reflectionsLayout.css">

The media attribute tells the browser to use this sylesheet for all media types, i.e., for screens and printers. In the pages that are formattted to print is a stylesheet that cascades below the main stylesheet and therefore supercedes it. This stylesheet controls the printing. IE seems to have its own views on font size so we use some conditional comments to coax it to our way of thinking.

Here and there throughout the website are pages that contain onscreen navigation ("jump to top" and that sort of thing). We hide them when printing by saying class="navstrip" which you can see will result in those elements being hidden.

The specification of

<body onload="window.print()">

(all lower case for XHTML purposes) is what forces the print dialog to appear.

The remaining problem is how to specify CSS formatting for images so that text flows around them as we want. The formatting seems to work on screen for all browsers but only on some browsers for printing.

<style type="text/css" media="print">

  body        { margin: 0; padding: 0; width: 100%; }				
				
    #wrapper    { margin: 0; padding: 0; width: 100%; }
		
      #center     { margin: 0; padding: 0; }

      #content    { font-size: 11pt; line-height: 100%; font-family: "Times New Roman", Times, serif; }

        .navstrip   { visibility: hidden; }
				
</style>

<!--[if IE]>
<style type="text/css" media="print">

      #content    { font-size: 14pt; }
				
</style>
<![endif]-->

(my thanks to http://centricle.com/tools/html-entities/ for HTML encoding)

Floating Three-Column CSS Layout

A current fad in web page styling is to use CSS exclusively to define the basic page sections. The "old" way of doing this was to use tables, but that's no longer stylish. Instead, we are exhorted to use CSS exclusively.

A very common page layout has a head and a foot with three columns sandwiched in between. Philadelphia Reflections uses this layout.

Most descriptions of this layout style that I have found Googling around the Internet involve absolute positioning which very often does not adapt well to differing screen sizes and browser window sizes. What we use here makes use of floating columns, which re-size themselves very nicely.

Several anomalies and quirks should be noted:

These quirks and anaomalies make me think that maybe this either isn't quite kosher or else may be superceded by later CSS definitions. But for the time being, this works very happily and both the HTML and the CSS validate perfectly well.

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
  <head>
    <title> Floating Three-column CSS example</title>

    <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">

<style type="text/css">


 #head {
  background-color:blue;
  color:white;
  text-align:center;
  }

 #wrap {
  }

 #left {
  float:left;
  width: 30%;
  }

 #right {
  float:right;
  width:30%;
  }

 #center {
  }

 #clear {
  clear:both;
  }

 #foot {
  background-color:red;
  color:white;
  text-align:center;
  }

</style>
  </head>

  <body>

    <div id="head">
      <p>Head</p>
    </div>

    <div id="wrap">

      <div id="left">
        <p>Left</p>
      </div>

      <div id="right">
        <p>Right</p>
      </div>

      <div id="center">
        <p>Center</p>
      </div>

      <div id="clear"></div>

    </div>

    <div id="foot">
      <p>Foot</p>
    </div>

  </body>
</html>

Open a new window with XHTML

Once upon a time you could say

<a href="link" target="_blank">Click to open a new window</a>

and a new window would open. Highly annoying if used very often, but sometimes it's the right thing to do.

And then XHTML comes along and this is not longer legal.

target="_blank" is "deprecated" without a single word as to what a poor developer is to substitute.

Here's what you do:

<a href="link" onclick="window.open(this.href); return false;">Click to open a new window</a>

DHTML, PHP and MySQL References

Start with

DHTML and CSS for the World Wide Web:
Visual QuickStart Guide
by Jason Cranford Teague

DHTML = HTML + CSS + DOM + JavaScript

This book will get you up and running quickly with client-side programming.

Then you need to learn server-side programming.

PHP is an open source server-side scripting language that is easy to learn and very powerful. MySQL is the same ... open source relational database.

The text book for these technologies is

PHP and MySQL for Dynamic Web Sites:
Visual QuickPro Guide
by Larry Ullman

Master those two books and you'll be creating very powerful scripts on both the client and server side that produce dynamic and elegant results.

While you're in the process of doing this, you will constantly need to reference manuals for syntax, functions, etc. There are many, but two will suffice for 90% of what you need:

PHP Manual

W3 Schools

Web hosting providers

Choosing a web hosting service provider is difficult. There's no brand to rely on and an Internet search turns up confusing claims and offers. We have used two providers:

Based on our experience, Internet Planners is a reasonable choice for web hosting; Network Solutions is a bad choice.

Regular Expressions

Anyone who has used the expression *.doc to search for Word files has used Regular Expressions ("regex") without realizing it. Regex arose from mathematical theory and is available in many programming languages; it is simply the only way to deal with large amounts of text. And yet most people are completely unaware of it.

Philadelphia Reflections uses regex extensively for two primary purposes: (1) checking input from forms and (2) modifying HTML input in during the creation of articles for the site.

The text PHP and MySQL by Larry Ullman has a very good introduction to regex in his chapter on security.

The great advantage of regex is that it can identify very complex patterns in a mass of text. The great disadvantage of regex is that it has developed in sort of an underground way and there exist numerous varieties that are essentially incompatible. PHP offers two regex functions: one for the POSIX Extended variety of regex and he other for the Perl language compatible vesion called PCRE. POSIX is less powerful but far easier to learn. JavaScript offers its own variety of regex which isn't quite the same as either of the two PHP versions.

References include the Ullman book, the PHP online manual has a number of handy tips on regex use in its two supported varieties, the O'Reilly book Mastering Regular Expressions is interesting and Jan Goyvaerts has a very helpful website (http://www.regular-expressions.info/) and book Regular Expressions: The Complete Tutorial.

My experience is that this area requires diligent hacking which may be sub optimal but unavoidable ... for this purpose, Jan Goyvaerts' Regex Buddy is indispensible; you simply must get this program if you hope to make anything of Regex.

Here are examples of checking for a valid email address in both Javascript and PHP:

Javascript


// check email

var namePattern = /^[a-zA-Z0-9][a-z0-9_.-]*@[a-z0-9.-]+\.[a-z]{2,4}$/i;

document.comment_form.email.value = trim(document.comment_form.email.value);
					
if	(
	(  document.comment_form.email.value.length > 0)
			&&
	(! document.comment_form.email.value == "[none]")
			&&
	(! document.comment_form.email.value.match(namePattern))
	) 
		{
			alert("Please enter a valid email address");
    			document.comment_form.email.focus();
       			document.comment_form.email.select();
			problem = "yes";
       			return false;
   		}

PHP


// check email

$emailpattern = "^[a-zA-Z0-9][a-z0-9_.-]*@[a-z0-9.-]+\.[a-z]{2,4}$";
		
if	(
	(trim(strlen($_POST['email']))  > 0)
		
			and
					
	(!$_POST['email'] == "[none]")
			
			and

	(!eregi ($emailpattern, stripslashes(trim($_POST['email']))))
	)
		{
		$inputerror		=	TRUE;
		$inputerrormessage	.=	"<br />* An invalid email address was entered";
		}


Incomprehensible? Yes, absolutely.

Useful? More than you can realize until you are actually faced with the problem of, say, verifying that a user has input a valid email format, or trying to figure out whether a user-input IMG tag is using the correct syntax; or else maybe trying to convert a huge web page from XHTML 1.1 to HTML 4.01 because you've determined that the browser is syntactically crippled.

And, once you get deep into it, the stuff is actually intriguing and fun.

Javascript: document.write and XHTML

For reasons that make no sense to me, the Javascript command document.write does not work when your page is rendered properly in XHMTL (as described elsewhere in this Topic).

I have searched the web in vain to find a Javascript solution. Many are offered but none work worth a damn.

So don't bother. Use PHP's echo function. It works perfectly.

Captcha

With the rise of spam entries in web forms, a security feature called "captcha" has been developed.

CAPTCHA stands for "Completely Automated Public Turing test to tell Computers and Humans Apart". The idea is that only a human could read the letters contained in the image and then enter them in the form. "Accessibility", ie., designing websites to accommodate people with handicaps is obviously hindered by Captcha; but at least given our experience with this website, spamming is a huge problem and the inability of handicapped people to leave comments is a price we are willing to pay to rid ourselves of spam. The W3C, the Godhead of web standards, does not agree with me and lectures at length on the futility of captcha: Inaccessibility of CAPTCHA. Whatever. I may get around to implementing some of their recommendations later, if we continue to be spammed.

Spammers have countered captcha in a number of ways. The first is OCR, which is why the images have fuzzy backgrounds and distorted letters: trying to defeat OCR programs. As OCR techniques have improved, captcha programs have moved from letters to "objects" such as kittens, boxes, etc., which are thought to be harder for computers to recognize; harder for people, too: cat vs kitten, for example. I am amazed to learn during my captcha research that there are spammers who offer micro-payments to people in India, etc. to enter hundreds of spam manually in captcha-ed websites that have defeated their automated spamming systems. Move, counter move; seemingly endlessly.

In this website captcha has been implemented using PHP: the comments form that appears at the end of every page has an image created using the PHP image-creation routines which has random characters in it. If the characters in the image are entered correctly in the form, the comments are entered into the database.

I cribbed the PHP captcha code from http://www.white-hat-web-design.co.uk/articles/php-captcha.php and it worked right out of the box with the minor exception that the form HTML didn't quite pass XHTML muster; easily fixed. (I have subsequently discovered that PHP security and sessions don't play well together; this problem remains unresolved and I've had to turn off captcha processing for my secure pages.)

I implemented a number of other spam counter measures before I got around to captcha, which involved noticing what the spammers did and writing code to frustrate it. I am constantly on the lookout for new security techniques to implement.

RSS, Atom, Syndication, etc.

The world is full of XML and XML-like file formats for syndication purposes

Here's the list of files we generate automatically for submission to search engines and such.

(For right now, things are a bit abbreviated)

http://www.philadelphia-reflections.com/reflectionsRSS.xml (RSS Syndication file)
http://www.philadelphia-reflections.com/reflectionsATOM.xml (Atom Syndication file)
http://www.philadelphia-reflections.com/sitemap.xml (Google sitemap)
http://www.philadelphia-reflections.com/siteinfo.xml (A9/Amazon siteinfo.xml)
http://www.philadelphia-reflections.com/reflectionsIDIF1.xml (Yahoo IDIF file 1)
http://www.philadelphia-reflections.com/reflectionsIDIF2.xml (Yahoo IDIF file 2)
http://www.philadelphia-reflections.com/reflectionsIDIF3.xml (Yahoo IDIF file 3)
http://www.philadelphia-reflections.com/reflectionsIDIF4.xml (Yahoo IDIF file 4)
http://www.philadelphia-reflections.com/reflectionsIDIF5.xml (Yahoo IDIF file 5)
http://www.philadelphia-reflections.com/IDIFpointer.txt (Yahoo IDIF pointer file)
http://www.philadelphia-reflections.com/urllist.txt (Yahoo urllist.txt)

Validate Short RSS | The Short RSS File itself
Validate Short rss (lower case) | The Short RSS File itself (lower case)
Validate Short ATOM | The Short ATOM File itself

Weblogs.com extended successfully pinged
Weblogs.com successfully pinged
blo.gs successfully pinged
Technorati successfully pinged
Ping-O-Matic successfully pinged
Syndic8 successfully pinged (Feed ID 477463)

Ping Blogroller manually
Ping MyYahoo manually



The RSS and Atom validator (http://feedvalidator.org/) has a length restriction. I don't know what it is, exactly, but it bombs if your file is "too long". Since most syndication readers run the validator before they'll accept a feed, I have resorted to creating a short file, which is what I point to in my meta tags.



Here's how I provide change frequency and priority for our Google sitemap (in PHP ... $mod is the variable containing the date last modified)

$GOOGLEpriority = "0.0"; $GOOGLEfreq = "yearly";	// default

if ($mod > mktime(0,0,0) - 86400*210)	{$GOOGLEpriority = "0.1"; $GOOGLEfreq = "monthly";}	// past 210 days
if ($mod > mktime(0,0,0) - 86400*180)	{$GOOGLEpriority = "0.2"; $GOOGLEfreq = "monthly";}	// past 180 days
if ($mod > mktime(0,0,0) - 86400*150)	{$GOOGLEpriority = "0.3"; $GOOGLEfreq = "monthly";}	// past 150 days
if ($mod > mktime(0,0,0) - 86400*120)	{$GOOGLEpriority = "0.4"; $GOOGLEfreq = "monthly";}	// past 120 days
if ($mod > mktime(0,0,0) - 86400*90)	{$GOOGLEpriority = "0.5"; $GOOGLEfreq = "monthly";}	// past 90 days
if ($mod > mktime(0,0,0) - 86400*60)	{$GOOGLEpriority = "0.6"; $GOOGLEfreq = "monthly";}	// past 60 days
if ($mod > mktime(0,0,0) - 86400*30)	{$GOOGLEpriority = "0.7"; $GOOGLEfreq = "monthly";}	// past 30 days
if ($mod > mktime(0,0,0) - 86400*7)	{$GOOGLEpriority = "0.8"; $GOOGLEfreq = "weekly";}	// past 7 days
if ($mod > mktime(0,0,0) - 86400)	{$GOOGLEpriority = "0.9"; $GOOGLEfreq = "daily";}	// yesterday
if ($GOOGLEmoddate == date("Y-m-d"))	{$GOOGLEpriority = "1.0"; $GOOGLEfreq = "hourly";}	// today



IDIF is a stupid format: it includes the entire blog_contents, so the files are huge. In the process of setting this up, I learned that flat files have a maximum size of 1.4 megs or so (the size of an old floppy disk), so I had to create more than one.

Which explains the stupid concept of a "pointer file"; instead of just giving Yahoo the IDIF file itself, you give it a pointer file with URLs pointing to the multitude of IDIF files. Really stupid.

News flash, after finding the Journal Of Ovid on the web, I learned about length restrictions for the input fields (described below). This information was not contained on the Yahoo web site describing their file format. It considerably reduced the file sizes but I retained the structure of multiple files because who knows what I'll learn next?

IDIF title must be a maximum of 80 characters
IDIF description must be a maximum of 180 characters
IDIF body must be a maximum of 1000 characters
I'm only guessing about keywords

Thanks to the Journal Of Ovid on the web for this secret information

From the inside out: trim, replace whitespace (thanks to the PHP manual for this), shorten to maximum length

$IDIFtitle		= substr( preg_replace ('/\s\s+/', ' ', trim($title) ), 0, 80 );
$IDIFdescription	= substr( preg_replace ('/\s\s+/', ' ', trim($description) ), 0, 180 );
$IDIFkeywords		= substr( preg_replace ('/\s\s+/', ' ', trim($keywords) ), 0, 79 ) . " ";
$IDIFblog_contents	= substr( preg_replace ('/\s\s+/', ' ', trim($blog_contents) ), 0, 1000 );

Yahoo is said to support a simple text file list of URLs "urllist.txt" Documentation, of course, is scarce


(my thanks to http://centricle.com/tools/html-entities/ for HTML encoding)

Web Standards Validation

There are two primary aspects of a website that need validation:



1. (X)HTML

You can use the W3C's QA Markup Validation Service.
The URL to test the main page of Philadelphia Reflections is http://validator.w3.org/

Firefox has several useful add-ons for (X)HTML validation; one that uses Tidy is here: Html Validator

2. CSS

The W3C has a validation service for CSS, too.
For Philadelphia Reflections, the following URL checks all the CSS definitions in the main page: http://jigsaw.w3.org/css-validator/ (note: this validator is a little flakey: it produces different answers for the same file; you have to refresh a couple of times to get the whole story)

Firefox has several useful web developer add-on tools; try this one: Web Developer


Once you've gotten the HTML and CSS basics under control, there are other aspects of your site that you will want to validate:

Broken Links

The W3C will check all your links for both response time and validity.
http://validator.w3.org/checklink/checklink

Tidy

There is an absolutely lovely program called HTML Tidy, origianlly written by Dave Raggett and decribed by the W3C here: http://www.w3.org/People/Raggett/tidy/

Calls to Tidy are available in some newer renditions of PHP (sadly, not the one we are using), however, on Widows (only) versions of Firefox and Mozilla, you can download an extension that will provide all the Tidy functions in your browser! ... https://addons.mozilla.org/firefox/249/. This a fantastic feature that I use all the time.

Syndication XML Validation

Validating RSS and Atom files is greatly facilitated by http://feedvalidator.org/. It has a number of quirks, the worst of which is that it has a length limitation that we exceed and so we have to provide "short" syndication files since all the feed aggregators use this facility and reject any feeds that aren't validated by it.

Google Sitemap Validation

If you submit a sitemap to Google through their Webmaster Tools facility they will validate your sitemap when they load it. An external validation tool is available here: Validome Google Sitemap(s) Validator

Yahoo and Microsoft have agreed to support Google's Sitemap protocol and to support the inclusion of the line "Sitemap: http://www.philadelphia-reflections.com/sitemap.xml" in robots.txt. If other search engines adopt this facility it will make it much easier to get into the world's many search engines ... they'll pick up this line instead of us having to hunt them down.

Meta Tag Validator

As you puzzle the mysteries of search-engine indexing, you'll want to check your meta tags: http://www.widexl.com/remote/search-engines/metatag-analyzer.html

gzip Compression & Headers

When you start getting really fancy and want to include automatic gzip compression, you'll want to see it in action and you'll want to check out all of your HTTP headers: http://www.gidnetwork.com/tools/gzip-test.php

Response Time

Of course, the reason you''re experimenting with gzip is because you're concerned about response time.
(1) Try this site for a detailed analysis: http://www.websitepulse.com/help/tools.php
(2) Firefox to the rescue again: FasterFox is another lovely add-in: https://addons.mozilla.org/firefox/1269/

Geo Tags

Check the validity of your geo tags here: Geo-Tag Validator

Big List

http://uitest.com/en/analysis/ is the mother of all lists of validations routines

Ampersand Madness: Convert & to &amp; to prevent XHTML errors

The whole subject of "encoding" gives me a headache.



Encoding In General

The first thing you have to know is: what is HTML encoding ... so look here:
http://htmlhelp.com/reference/html40/entities/
or here:
http://www.cookwood.com/html/extras/entities.html

(These are HTML encodings; URL encoding is something else again ... look here:
» http://www.blooberry.com/indexdot/html/topics/urlencoding.htm)

Ampersand Encoding and Conversion

Later on, you'll find out that the ampersand is a huge source of XHTML errors because it has to be written

but you will struggle endlessly with how to get the darn thing to stay converted. First of all, content providers feel justifiably justified in including bare naked "&"s wherever they please; second of all, you will find that encoded ampersands get stripped back to their bare naked selves by browsers and other well-meaning sorts.

So, my undying thanks to Michael Ash's Regex Blog for providing the regex pattern in the following bit of PHP code:


$pattern = '/&(?!(?i:\#((x([\dA-F]){1,5})|(104857[0-5]|10485[0-6]\d|1048[0-4]\d\d|104[0-7]\d{3}|10[0-3]\d{4}|0?\d{1,6}))|([A-Za-z\d.]{2,31}));)/i';
				
$replacement = '&amp;';
				
$string = preg_replace ( $pattern, $replacement, $string);

I don't know how it can possibly work, and I may yet eat my words, but for the moment it seems to do the trick.

Ampersand Encoding In RSS

Another thing: &#x26; is the only ampersand encoding form acceptable to both RSS and Atom. So, look at the souce of this page and you will find that I use this encoding in the title ... that's because the title goes into the Title field of my RSS and Atom feeds.

CSS Zen Garden Suggestions

Here is a list of links (that open in their own pages) that show some of my favorite web designs. The CSS Zen Garden is a website that illustrates what can be done with clever CSS design. The HTML and the content are exactly the same in each of these links, only the CSS changes; but what a difference!


C Note

Dark Rose

Dead or Alive

Egyptian Dawn

Invitation

Mediterranean

Mozart

Odyssey

Tranquille

Zen Pool

webZine

White Lily

Another website to consider is http://www.freecsstemplates.org/

Font Families

The following link shows the results of a survey done to find out which font families are installed on Windows machines. This should help determine which fonts to use.

http://www.codestyle.org/css/font-family/sampler-WindowsResults.shtml

Identifont is a site that helps identify good font choices.

Emails From Iraq (2)

This is my third draft of this issue. In the first draft, I ended with what seemed a logical afterthought. Soon after writing it, I received a list of questions from a friend and the afterthought became a purposeful forethought. Two of the questions provoked thought and meditation. One question was, "What surprised you the most?" The other was, "What scares you the most?" Interestingly, the answer to both questions is, "Reality." The third draft was necessary after learning where I am being sent and what restrictions there are on my words.

As an alumnus of Bosnia, I thought my experience as a civilian contractor in a war zone prepared me for this. The truth is working in a war zone 4 years after a war offers little preparation for working in an active war zone. I experienced the war more than once this week, including explosions and smoke, warning and all-clear sirens, orders to get inside, gunfire, and incoming and outgoing shells, in addition to the 24/7 roar of military helicopters directly overhead. As in the past, nothing hit our camp but there is little more to say than "Wow!" We are advised to find ways to deal with stress, and days like these are the reason why. I have so much to learn.

I originally started this report with the intention of providing a few details about the project only to learn that constraints on disseminating information are very, very strict. So until I learn exactly what I can and cannot say, I will refrain from offering much about where I am, what I am doing and how it is getting done. Some of my friends think I work for the CIA. Sexy idea but untrue! The real reason is to protect everyone's ability to work in Iraq safely.

If you want to read an informative, published report about the project please go to http://www.usip.org/pubs/specialreports/sr185.html - particularly pages 6-7. I am a member of a Provincial Reconstruction Team (PRT) - Governance (RTI - contractor). This is part of President Bush's "New Way Forward." Without revealing my views of our president, I am allowing my perspective on Iraq to crystallize independently of US media - a personal objective. I can tell you the optimism here is as strong as the pessimism there. I learned in other developing countries, and it is certainly true here, that progress is measured in millimeters. In a broad sense, are we making a difference? Yes. Are we making mistakes? Yes. Is this expensive? Yes. Is money being wasted? Only as artists waste raw materials. Should we pull out or set a date for withdrawal? No. Should we have come here in the first place? I don't know. All the effort, cost and waste could have been avoided if we had never come here in the first place, regardless the arguments about WMD's. Having been part of reform of the former Soviet Union, however, I will not agree that coming here was a mistake - at least not now.

I learned today where I will be stationed, and I also received clear direction about what can and cannot be reported. If you did not read the cover email carefully, please do so. My ability to continue sending these reports depends entirely on your conformity to the rules. My assignment is in Anbar Province, the largest province (yet sparsely populated) to the west of Baghdad. Anbar is predominantly Sunni and borders Syria, Jordan and Saudi Arabia and is home to Fallujah, Ramadi and Abu Ghraib, among other less notable locations. The good news is I will live on a Marine Base in Ramadi. Anbar is where much of the surge is taking place, the latest attempt to restrict the insurgency. I understand a goal of this surge is to bring peace and stability to a predominantly Sunni area as balance to the Shia dominance. You can learn more about Anbar Province by looking at http://en.wikipedia.org/wiki/Al_Anbar_Governorate. More on this later.

I read in an online newspaper this morning about fighting yesterday in a city south of Baghdad. This must have been what we heard. We are 8 hours ahead of the east coast.

Geo Positioning

Geo Tagging refers to adding latitude and longitude information to websites and photographs. This has been around for a long time but it has taken the advent of Google Earth for it to really start to catch on.

This blog entry has geo meta tags that you can see if you look at the HTML source ("View > [Page] Source"). The input was as follows:

Address: 82 Devonshire St Boston MA

Lat: 42.3578 Lon: -71.0577

Descriptive Place Name: Fidelity Investments headquarters

Region: US-MA Country Code: US Country Name: United States


This creates meta tags in the HTML Header as follows:

<!-- geo tags for 82 Devonshire St Boston MA -->
<meta name="ICBM"          content="42.3578, -71.0577" />

<meta name="geo.country"   content="US" />
<meta name="geo.region"    content="US-MA" />
<meta name="geo.placename" content="Fidelity Investments headquarters" />
<meta name="geo.position"  content="42.3578; -71.0577" />

<meta name="tgn.name"      content="Fidelity Investments headquarters" />
<meta name="tgn.nation"    content="United States" />

The Region, Country Code and Country Name can be found here: ISO-3166-1 Country Names

geo.placename and tgn.name are often rendered as the city name but are intended to describe the geographical feature ("Pyramids of Giza" or something). This tag is optional.

HTML geo meta tags can be validated here: {geo tag validation}



There is a search engine of long standing that reads HTML geo meta tags and indexes the website based upon its location; for searching, it groups sites based on their geographic proximity: GeoURL.



Photographs can also contain geo meta data, so-called EXIF data (Firefox has an EXIF viewer AddOn).

JPEG is the most common image format and the easiest to deal with. The combination of Picassa2 and Google Earth allow you easily to add this information to your own photos.

The process of adding lat and lon to your photographs is this:

1. Select one or more photos in Picassa
2. Select Tools > GeoTag > GeoTag with Google Earth ...
3. This starts Google Earth and you can "fly" to the location of the picture

4. A small Picasa window will appear in Earth's lower-right corner displaying thumbnails of the pictures you selected; press the "Geotag" button.
5. When all of your pictures are tagged, press the "Done" button

Slowly, camera manufacturers are providing GPS capability. Some few have GPS devices built in and some others allow an external GPS device to be attached, although both Canon and Nikon are way behind the curve ... if you own either, you can essentially forget it: the best - lousy - solution is to carry around a GPS with you and synchronize the times ... ugly.


The Google Maps API allows maps to be embedded in a website as is done here. Google Maps API

The JavaScript required to embed the map on this page can also be seen in the HTML source ("View > [Page] Source"). In addition to JavaScript, you need a DIV with an ID of "map" or whatever is specified in the JavaScript document.getElementById entry, which specifies the height and width of the map to be displayed.

To embed these maps you must register with Google


In addition, there are extensions to ATOM and RSS to include lat and lon in your syndication feeds; there are three standards that I have found: GeoRSS (ATOM example) , W3C Geo (RSS example) and an "ICBM RSS Module". This website extends the namespaces of both its ATOM feed and its RSS feed to include all the tags.

Google, Yahoo and Microsoft all now support GeoRSS as a feed to their map programs. My sense of it is that KML is a richer protocol, allowing more features, but fundamentally all these XML variants do mostly the same thing.

Google Sitemaps can include links to KML files (and ATOM, now, too). Part of the sitemap generation on this site is some code that picks up every *.kml and *.kmz file in the /kml/ folder and adds them to our sitemap.xml file.


Google Earth is filled with delights, not the least of which is a Flight Simulator! Google Earth Flight Simulator Keyboard Controls


KML ( Keyhole Markup Language, Keyhole being the predecessor to Google Earth) is an XML protocol that allows you to incorporate Google Earth into graphical presentations. Google KML Overview

Google Earth Outreach helps you get started: Google Earth Outreach

An extraordinary collection of KML files you can view is found here: Spectacular satellite images of the world

I found a KML editor here: NorthGates' KML Editor for Windows. It's rudimentary but very handy for what it does do.

Here's the Google Earth tools list where I found the KML editor: EarthPlot Software Tools For Google Earth


The way we serve the KML in the link that connects to Google Earth from individual blogs uses the following PHP script as its base:

<?php

// See Google Earth's KML 2.1 Reference
// http://code.google.com/apis/kml/documentation/kml_tags_21.html

$lat			= $_GET['lat'];
$lon			= $_GET['lon'];
$placename		= $_GET['placename'];
	
$altitude		= $_GET['altitude'];
$range			= $_GET['range'];
$heading		= $_GET['heading'];
$tilt			= $_GET['tilt'];
	
if ($altitude	== NULL) {$altitude	= 0;}
if ($range	== NULL) {$range	= 1000;}
if ($heading	== NULL) {$heading	= 0;}
if ($tilt	== NULL) {$tilt		= 0;}
	
$description	= "<h3><font color=\"#ea9f20\"><a href=\"http://www.philadelphia-reflections.com/\">
		PHILADELPHIA REFLECTIONS</font></a></h3>
		<p>The musings of a Philadelphia physician who has served the community for six decades.</p>";
	
	
header('Content-Type: application/vnd.google-earth.kml+xml');
header('Content-Disposition: inline; filename="philadelphia-reflections.kml"');

echo '<?xml version="1.0" encoding="UTF-8"?>'; 

?>

<kml xmlns="http://earth.google.com/kml/2.0">

  <Placemark>
    <name><?php echo $placename; ?></name>
    <description>
        <![CDATA[<?php echo $description; ?>]]>
    </description>
    
    <LookAt>
      <longitude><?php echo $lon; ?></longitude>
      <latitude><?php echo $lat; ?></latitude>
      
      <altitude><?php echo $altitude; ?></altitude>
      <range><?php echo $range; ?></range>
      <tilt><?php echo $tilt; ?></tilt>
      <heading><?php echo $heading; ?></heading>
      
      <altitudeMode>relativeToGround</altitudeMode>
    </LookAt>
    
    <Point>
      <coordinates><?php echo "$lon,$lat,$altitude"; ?></coordinates>
    </Point>
    
  </Placemark>

</kml>


Of course, GPS devices are an integral part of this process of Geo Positioning. GPS devices are supposed to support the open-source protocol GPX,
which is an XML-based description of waypoints and routes. Wikipedia describes GPX here: GPS eXchange Format

The GPX protocol's official website is here: GPX The GPS Exchange Format

Google Earth supports raw GPX (File > Open ...) and when you open a GPX file in Google Earth, it converts it to KML. But if you want stand alone programs to do this:

If you have non-standard GPS data, you may want to have a look at GPS Babel for conversion of native GPS formats as well as the GPS Utility and G7ToWin

A nice blog on these things relative to Google Maps is here: Using XSL to Transform Google Earth (KML) and GPX to Google Maps API


At Philadelphia Reflections, we are creating tours by carrying a GPS and a camera around on our travels. The GPS track becomes a path and waypoints become placemarks. When you come home, download the GPS data in GPX format and open up the GPX file in Google Earth. Use Google Earth to edit the placemark balloons, including pictures and text.



There are many, many sightseeing blogs around that take you to interesting places on Google Maps and Google Earth. A place to start looking is Sightseeing with Google Satellite Maps


Somehow, the concept of "mashup" is related to all of this but it sort of sounds like the term "multimedia" a few years ago ... fancy in concept but somewhat vague in reality.

Google has a Mashup Editor and Wikipedia has a definition but it's not clear what it all adds up to.



(my thanks to http://centricle.com/tools/html-entities/ for HTML encoding)

Process .htm and .html as php

It is sometimes helpful to include php scripting in files that do not have the file extension of php.

There is quite a lot of discussion on the web about this but at least on this server the answer is not what most people think.

In the .htaccess file in the root folder include these two lines:

AddType application/x-httpd-php .html .php .htm
AddHandler application/x-httpd-php .html .php

Call KML files from within a blog or topic

Here's how to create a button in your blogs or topics that calls KML or KMZ files you create. In the Modify A Blog utility you can include a KML or KMZ file, but to call it explicity from within the blog, you can create a button as shown here.

  1. Create and save your KML file.
  2. FTP the file to the kml folder in Philadelphia Reflections
  3. Use the following code in a blog or topic to call the kml file


<button onclick="location.href='http://www.philadelphia-reflections.com/kml-read.php?file=Franklin.kmz'">Button Label</button>

To create this button:



Instead of Franklin.kmz, put any .kml or .kmz file that is in the kml folder:
http://www.philadelphia-reflections.com/kml/

Send a KML file from disk using PHP

Sending a kml or kmz disk file is as easy as clicking on it. But different browsers react differently, some asking you which program to use others storing the file on your desk top, etc. Preprocessing the file through PHP can reduce some of these annoyances.

<?php

//
// reads and sends a kml or kmz file
// located in /whatever/kml/
//
// calling protocol:
// this-program.php?file=somefile.kml
//

// read the input and check that it's a kmz or kml file
// ....................................................

$kml_file	= $_GET['file'];

if (($kml_file === NULL) or ($kml_file == "")) {exit ("error message");}
		
if ((substr($kml_file, -4) != ".kmz") AND (substr($kml_file, -4) != ".kml"))	
	{
	exit ("error message");
	}


// prepend the file path information to the file name and check that it exists
// ...........................................................................

$kml_file_name = "/whatever/kml/" . $kml_file;
	
if (!file_exists($kml_file_name)) { die ("error message");}


// send out the HTTP header information followed by the file contents
// ..................................................................
	
header("Cache-Control: no-cache, no-store, must-revalidate"); // trying to keep from getting the files stored on the local computer
header("Expires: Mon, 26 Jul 1997 05:00:00 GMT");

if (substr($kml_file, -4) == ".kml") {header('Content-Type: application/vnd.google-earth.kml+xml');}
if (substr($kml_file, -4) == ".kmz") {header('Content-Type: application/vnd.google-earth.kmz');}

header("Content-Disposition: inline");
header("Content-Description: KML or KMZ data intended for Google Earth");

readfile ($kml_file_name);

?>

BU to Pier 7 GPS Tour

Example of Including a KML file using the GPS tour from BU to Pier 7

Static vs Dynamic URLs

It used to be that no spiders or search engines could index a dynamic URL, namely one that contained a "?" followed by parameters to be used by PHP, ASP or other server-side scripting languages to drive a website using a database.

Nowadays, Google and Yahoo seem to do a perfectly fine job of indexing dynamic URLs but Google has a disclaimer warning that it may still encounter problems with dynamic URLs and the SEO literature is still full of warnings that other spiders and search engines may be blind to everything to the right of the "?".

Furthermore, a *.php extension is an invitation to bad guys to try to break in and wreak many sorts of havoc: this site was hacked by Nigerians a few years ago using PHP tricks and they managed to use it as an email factory until our ISP shut us down. I came on the scene at that point and implemented every safeguard I could find, but the concern still lingers.

Finally, dynamic URLs are not user friendly ... human beings generally do not know what to make of long strings of obscure parameters.

Apache has a feature called "mod_rewrite" that allows you to specify, via regex, that you want incoming URLs to be transformed in some way. Apache's instructions on this subject are here: URL Rewriting Guide. I have used that facility at Philadelphia Reflections to use static URLs for public use while still allowing me to use parameters to drive the website with the database.

Two excellent articles on this got me started:

Here's what I did:

Step 1: htaccess

I added these lines to the htaccess file

Options +FollowSymLinks
RewriteEngine on
RewriteRule ^(blog|topic)/([0-9]+)\.html?$ reflections.php?type=$1&key=$2
  1. ^(blog|topic)/([0-9]+)\.html?$ is the pattern to be compared against all incoming URLs.

    If matched, it changes the URL to reflections.php?type=$1&key=$2

  2. The ^ ... $ sequence in the pattern says that we will match the whole string, not just some part in the middle
     
  3. (blog|topic)/ matches either   "blog"   or   "topic"   followed by   "/"  
     
  4. ([0-9]+) matches one or more digits
     
  5. \.htm matches   ".htm"  
     
  6. l? matches 0 or 1 lower-case Ls (so that we will match either htm or html)
     
  7. In the replacement string $1 is replaced with the contents of the first () in the pattern: either "blog" or "topic"
    ... and $2 is replaced by the second () in the pattern, namely the numeric ID on the database of the blog or topic

The result is that

http://www.philadelphia-reflections.com/blog/906.htm

is transformed into

http://www.philadelphia-reflections.com/reflections.php?type=blog&key=906

The latter is what is passed in to me in the reflections.php routine, which tells me to pull up blog #906 from the database.

Both of these URLs are equivalent to the old, ugly dynamic URL

http://www.philadelphia-reflections.com/reflections.php?content=blogs_alpha/zmadame_butterfly.html

which still works, in case there are any legacy bookmarks or links out there, but going forward the new, simple, static URL is the face we will present to the world.

Step 2. SMOP

After the htaccess regex was debugged, all that was left was a simple matter of programming. In fact, I had to completely rewrite the driver script, reflections.php, and the XML creation script which creates the RSS, sitemap, etc. files; plus a lot more besides. It was a lot of work but the breakthrough was in figuring out the htaccess trick; everything else was just work.


In July 2008, after Volumes were implemented, another RewriteRule was implemented:

RewriteRule ^volumes?/([0-9]+)\.html?$ volume.php?table_key=$1
Converts
http://www.philadelphia-reflections.com/volumes/3.htm
into
http://www.philadelphia-reflections.com/volume.php?table_key=3

Round the Block in Haddonfield

Google maps themseves contain a certain amount of innacuracy, so precision mapping of this walk around the block gives a somewhat ragged picture.

HTML Forms

How do you (a) open a form when a radio button is clicked (b) in a new window?

Here's how it's done on this website.

<html>
<head>

<script type="text/javascript">

	/* javascript function called by the radio buttons 
	     to submit the form when clicked */

	function formSubmit()
	{
	document.getElementById("form_x").submit()
	}
	
</script>

</head>
<body>

  <form name="form_x" id="form_x"
	action="some_routine.php"
	target="newIMGwin"
	method="post" 
	style="whatever">

	<fieldset>
	<legend>legend surrounding the form</legend>

	<input type="radio" name="key" value="1269" onclick="formSubmit()" />

	</fieldset>
  </form>

</body>
</html>

(my thanks to http://centricle.com/tools/html-entities/ for HTML encoding)

Regex URL Matching

On this site we check for the existence of a URL whenever an entry is updated

There are two key technologies at work


function url_exists($url) 
{
// 
// checks whether a URL actually exists on the Internet
//
$handle   = curl_init($url);
if (false === $handle)
   {
    return false;
   }
curl_setopt($handle, CURLOPT_HEADER, false);
curl_setopt($handle, CURLOPT_FAILONERROR, true); 
curl_setopt($handle, CURLOPT_NOBODY, true);
curl_setopt($handle, CURLOPT_RETURNTRANSFER, false);
$connectable = curl_exec($handle);
curl_close($handle);   
return $connectable;
}


function aExists($matches)
{
//
// function called by preg_replace_callback
//
// $matches[0] is the complete match
// $matches[1] the match for the first subpattern
//	enclosed in '(...)' and so on

//
// checks to see if a regular link exists
// something similar is done for img src= also
//

$srcURL = $matches[3];
		
if (url_exists($srcURL)) {do something; return "";}  
else {do something else; return "";}
}

$foo = preg_replace_callback(
            '/(.*?)(<a .*?href=")([^"]*)("[^>]*>)(.*?)(<\/a>)/i',
            "aExists",
            $source_string);

(my thanks to http://centricle.com/tools/html-entities/ for HTML encoding)

Parsing name-value pair attributes in an HTML tag

Not only do the attributes in an HTML tag come in random order but many are optional

Here's a regex solution:

<?php
function tagAttr($matches) {print_r($matches);}

$string = '<img src="/images/picture.jpg" width="300" class="left" alt="alt keywords" />';

$foo	= preg_replace_callback(
'/<img\b(?>\s+(?:alt="([^"]*)"|class="([^"]*)"|style="([^"]*)"|src="([^"]*)"|height="([^"]*)"|width="([^"]*)")|[^\s>]+|\s+)*>/i',
"tagAttr",
$string);
?>

Produces the following:

Array
(
    [0] => <img src="/images/picture.jpg" width="300" class="left" alt="alt keywords" />
    [1] => alt keywords
    [2] => left
    [3] => 
    [4] => /images/picture.jpg
    [5] => 
    [6] => 300
)

The regex is a series of alternating sequences; so, add href="([^"]*)"| in front of alt="([^"]*)" to select an additional attribute.

$matches[0] is the complete match
$matches[1] is alt=
$matches[2] is class=
$matches[3] is style=
$matches[4] is src=
$matches[5] is height=
$matches[6] is width=

My thanks (a) to Flagrant Badassery for putting me onto the idea and (b) to http://centricle.com/tools/html-entities/ for HTML encoding

Health Expenditures as a percent of GDP

My thanks to the US Census Bureau for this information.

The 2008 Statistical Abstract/International Statistics: Vital Statistics, Health, Education

http://www.census.gov/compendia/statab/cats/international_statistics/vital_statistics_health_education.html


Let's make a couple of comments. Health expenditure as a percent of GDP is a ratio. If health expenditures go up or down while GDP remains the same, the ratio goes up or down. But if GDP goes down or up and health expenditures remain the same, then the reverse is true. In the present atmosphere of slanted debate, America is being compared with France. If we compared the state of Mississippi with France, we would get a different opinion about France. If we compare the Ukraine with America, we get a different opinion about our expenditures. It really does seem more appropriate to compare America with Europe as a whole, rather than allow the comparison to be selective in its choice of the sections of Europe under comparison.

The point is this: if you add in the poorer sections of Europe, you reduce the average GDP of the nation (Europe) you compare with America, and hence you increase the expenditures for health care as a proportion of European GDP. At the same time, you would have to add in the average expenditures of the poorer sections of Europe, which will lower the average health expenditures per GDP unit. With twenty seven different countries included under the heading of "Europe", you can seem to illustrate just about any conclusion you intend to reach, just by modifying the mixture. The same is true of taking different states or sections of the USA. Minnesota is considerably cheaper than Florida, and the quality of health care is if anything superior.

Health officials have puzzled about this discrepancy between Minnesota and Florida for at least forty years, ever since Medicare costs were available to sort by zipcode. If we still can't understand the difference between Minnesota and Florida, there is little validity in drawing much health insurance conclusion from a comparison of USA data with that of France. Comparing USA data with the whole of the European community could result in some sort of opinion about national budgets, but nothing but quarrels if conclusions are reached about the nature of local health insurance. When we finally figure out what the same data about Minnesota and Florida means, perhaps that could be a starting place for further explorations. In case you haven't noticed, Minnesota and Florida share the same Medicare insurance.

Server-Side gzip Compression

Compression can reduce the size of the text (not images) of your web pages as they are transmitted outbound to the client. This will have only a small impact on response time over modern fiber connections but it will significantly reduce your bandwidth consumption (70% on average on this site.)

In XHTML vs. HTML I show how I implemented gzip compression on this site. The problem with that method is that it's a pain. So on another website I tried out the Apache htaccess method to instruct the server to compress all outbound pages. Works like a charm.


# See http://httpd.apache.org/docs/2.0/mod/mod_deflate.html

# Insert filter
SetOutputFilter DEFLATE

# Netscape 4.x has some problems...
BrowserMatch ^Mozilla/4 gzip-only-text/html

# Netscape 4.06-4.08 have some more problems
BrowserMatch ^Mozilla/4\.0[678] no-gzip

# MSIE masquerades as Netscape, but it is fine
# BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

# NOTE: Due to a bug in mod_setenvif up to Apache 2.0.48
# the above regex won't work. You can use the following
# workaround to get the desired effect:
BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html

# Don't compress images
SetEnvIfNoCase Request_URI \
\.(?:gif|jpe?g|png)$ no-gzip dont-vary

# Make sure proxies don't deliver the wrong content
Header append Vary User-Agent env=!dont-vary

Central Securities

SQL To Exclude A List Of Items

Let's say you have a table "Primary" that contains an "Email" field.

You would like to select all the email addresses in Primary except for the list of email addresses in the Email field in a table "Exclude".

This SQL will exclude the emails in "Primary" based on those contained in "Exclude".

SELECT * FROM Primary WHERE ((Primary.Email) Not In (SELECT Email FROM Exclude))

The Evolution of a Programmer

High School/Jr.High

  10 PRINT "HELLO WORLD"
  20 END

First year in College

  program Hello(input, output)
    begin
      writeln('Hello World')
    end.

Senior year in College

  (defun hello
    (print
      (cons 'Hello (list 'World))))

New professional

  #include <stdio.h>
  void main(void)
  {
    char *message[] = {"Hello ", "World"};
    int i;
 
    for(i = 0; i < 2; ++i)
      printf("%s", message[i]);
    printf("\n");
  }

Seasoned professional

  #include <iostream.h>
  #include <string.h>
 
  class string
  {
  private:
    int size;
    char *ptr;
 
  string() : size(0), ptr(new char[1]) { ptr[0] = 0; }
 
    string(const string &s) : size(s.size)
    {
      ptr = new char[size + 1];
      strcpy(ptr, s.ptr);
    }
 
    ~string()
    {
      delete [] ptr;
    }
 
    friend ostream &operator <<(ostream &, const string &);
    string &operator=(const char *);
  };
 
  ostream &operator<<(ostream &stream, const string &s)
  {
    return(stream << s.ptr);
  }
 
  string &string::operator=(const char *chrs)
  {
    if (this != &chrs)
    {
      delete [] ptr;
     size = strlen(chrs);
      ptr = new char[size + 1];
      strcpy(ptr, chrs);
    }
    return(*this);
  }
 
  int main()
  {
    string str;
 
    str = "Hello World";
    cout << str << endl;
 
    return(0);
  }

Master Programmer

  [
  uuid(2573F8F4-CFEE-101A-9A9F-00AA00342820)
  ]
  library LHello
  {
      // bring in the master library
      importlib("actimp.tlb");
      importlib("actexp.tlb");
 
      // bring in my interfaces
      #include "pshlo.idl"
 
      [
      uuid(2573F8F5-CFEE-101A-9A9F-00AA00342820)
      ]
      cotype THello
   {
   interface IHello;
   interface IPersistFile;
   };
  };
 
  [
  exe,
  uuid(2573F890-CFEE-101A-9A9F-00AA00342820)
  ]
  module CHelloLib
  {
 
      // some code related header files
      importheader(<windows.h>);
      importheader(<ole2.h>);
      importheader(<except.hxx>);
      importheader("pshlo.h");
      importheader("shlo.hxx");
      importheader("mycls.hxx");
 
      // needed typelibs
      importlib("actimp.tlb");
      importlib("actexp.tlb");
      importlib("thlo.tlb");
 
      [
      uuid(2573F891-CFEE-101A-9A9F-00AA00342820),
      aggregatable
      ]
      coclass CHello
   {
   cotype THello;
   };
  };
 
 
  #include "ipfix.hxx"
 
  extern HANDLE hEvent;
 
  class CHello : public CHelloBase
  {
  public:
      IPFIX(CLSID_CHello);
 
      CHello(IUnknown *pUnk);
      ~CHello();
 
      HRESULT  __stdcall PrintSz(LPWSTR pwszString);
 
  private:
      static int cObjRef;
  };
 
 
  #include <windows.h>
  #include <ole2.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include "thlo.h"
  #include "pshlo.h"
  #include "shlo.hxx"
  #include "mycls.hxx"
 
  int CHello::cObjRef = 0;
 
  CHello::CHello(IUnknown *pUnk) : CHelloBase(pUnk)
  {
      cObjRef++;
      return;
  }
 
  HRESULT  __stdcall  CHello::PrintSz(LPWSTR pwszString)
  {
      printf("%ws
", pwszString);
      return(ResultFromScode(S_OK));
  }
 
 
  CHello::~CHello(void)
  {
 
  // when the object count goes to zero, stop the server
  cObjRef--;
  if( cObjRef == 0 )
      PulseEvent(hEvent);
 
  return;
  }
 
  #include <windows.h>
  #include <ole2.h>
  #include "pshlo.h"
  #include "shlo.hxx"
  #include "mycls.hxx"
 
  HANDLE hEvent;
 
   int _cdecl main(
  int argc,
  char * argv[]
  ) {
  ULONG ulRef;
  DWORD dwRegistration;
  CHelloCF *pCF = new CHelloCF();
 
  hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);
 
  // Initialize the OLE libraries
  CoInitializeEx(NULL, COINIT_MULTITHREADED);
 
  CoRegisterClassObject(CLSID_CHello, pCF, CLSCTX_LOCAL_SERVER,
      REGCLS_MULTIPLEUSE, &dwRegistration);
 
  // wait on an event to stop
  WaitForSingleObject(hEvent, INFINITE);
 
  // revoke and release the class object
  CoRevokeClassObject(dwRegistration);
  ulRef = pCF->Release();
 
  // Tell OLE we are going away.
  CoUninitialize();
 
  return(0); }
 
  extern CLSID CLSID_CHello;
  extern UUID LIBID_CHelloLib;
 
  CLSID CLSID_CHello = { /* 2573F891-CFEE-101A-9A9F-00AA00342820 */
      0x2573F891,
      0xCFEE,
      0x101A,
      { 0x9A, 0x9F, 0x00, 0xAA, 0x00, 0x34, 0x28, 0x20 }
  };
 
  UUID LIBID_CHelloLib = { /* 2573F890-CFEE-101A-9A9F-00AA00342820 */
      0x2573F890,
      0xCFEE,
      0x101A,
      { 0x9A, 0x9F, 0x00, 0xAA, 0x00, 0x34, 0x28, 0x20 }
  };
 
  #include <windows.h>
  #include <ole2.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include "pshlo.h"
  #include "shlo.hxx"
  #include "clsid.h"
 
  int _cdecl main(
  int argc,
  char * argv[]
  ) {
  HRESULT  hRslt;
  IHello        *pHello;
  ULONG  ulCnt;
  IMoniker * pmk;
  WCHAR  wcsT[_MAX_PATH];
  WCHAR  wcsPath[2 * _MAX_PATH];
 
  // get object path
  wcsPath[0] = '\0';
  wcsT[0] = '\0';
  if( argc > 1) {
      mbstowcs(wcsPath, argv[1], strlen(argv[1]) + 1);
      wcsupr(wcsPath);
      }
  else {
      fprintf(stderr, "Object path must be specified\n");
      return(1);
      }
 
  // get print string
  if(argc > 2)
      mbstowcs(wcsT, argv[2], strlen(argv[2]) + 1);
  else
      wcscpy(wcsT, L"Hello World");
 
  printf("Linking to object %ws\n", wcsPath);
  printf("Text String %ws\n", wcsT);
 
  // Initialize the OLE libraries
  hRslt = CoInitializeEx(NULL, COINIT_MULTITHREADED);
 
  if(SUCCEEDED(hRslt)) {
 
 
      hRslt = CreateFileMoniker(wcsPath, &pmk);
      if(SUCCEEDED(hRslt))
   hRslt = BindMoniker(pmk, 0, IID_IHello, (void **)&pHello);
 
      if(SUCCEEDED(hRslt)) {
 
   // print a string out
   pHello->PrintSz(wcsT);
 
   Sleep(2000);
   ulCnt = pHello->Release();
   }
      else
   printf("Failure to connect, status: %lx", hRslt);
 
      // Tell OLE we are going away.
      CoUninitialize();
      }
 
  return(0);
  }

Apprentice Hacker

  #!/usr/local/bin/perl
  $msg="Hello, world.\n";
  if ($#ARGV >= 0) {
    while(defined($arg=shift(@ARGV))) {
      $outfilename = $arg;
      open(FILE, ">" . $outfilename) || die "Can't write $arg: $!\n";
      print (FILE $msg);
      close(FILE) || die "Can't close $arg: $!\n";
    }
  } else {
    print ($msg);
  }
  1;

Experienced Hacker

  #include <stdio.h>
  #define S "Hello, World\n"
  main(){exit(printf(S) == strlen(S) ? 0 : 1);}

Seasoned Hacker

  % cc -o a.out ~/src/misc/hw/hw.c
  % a.out

Guru Hacker

  % echo "Hello, world."

Thanks to http://www.ariel.com.au/jokes/The_Evolution_of_a_Programmer.html

HTML Anchor without an HREF?

Sometimes I want to execute a JavaScript function when a user clicks a link, but nothing else.

If I omit the href entirely, the cursor doesn't change and some browsers don't recognize the text as a link:

<a onclick="function();">

If I include the pound sign, which seems to be a very popular trick, I get sent to the top of the current page, which messes up both History and the backspace button; to say nothing of the fact that I don't want to jump to the top of the page:

<a href="#" onclick="function();">

Including the function in the href and omitting the onclick seems to be the answer to my specific problem:

<a href="javascript: function();">

Once again, my thanks to http://centricle.com/tools/html-entities/ for HTML encoding.

Retirement Planning Video

Debunking third-world myths with the best stats you've ever seen



Hans' website is here: http://www.gapminder.org/

His presentation software has been taken over by Google: Motion Chart

Income vs Life Expectancy over time

PHP script to display Google PageRank

Like so many things on this website, the code to find the Google PageRank of the pages was lifted from someone else's work. This work is particularly praiseworthy because it worked exactly as described the minute I got it implemented.

See PHP script to display Google PageRank


pagerank.php

<?php
define('GOOGLE_MAGIC', 0xE6359A60);
class pageRank{
var $pr; 
 function zeroFill($a, $b){
 $z = hexdec(80000000);
  if ($z & $a){
   $a = ($a>>1);
   $a &= (~$z);
   $a |= 0x40000000;
   $a = ($a>>($b-1));
  }else{
   $a = ($a>>$b);
  }
 return $a;
 } 
 
 function mix($a,$b,$c) {
   $a -= $b; $a -= $c; $a ^= ($this->zeroFill($c,13));
   $b -= $c; $b -= $a; $b ^= ($a<<8);
   $c -= $a; $c -= $b; $c ^= ($this->zeroFill($b,13));
   $a -= $b; $a -= $c; $a ^= ($this->zeroFill($c,12));
   $b -= $c; $b -= $a; $b ^= ($a<<16);
   $c -= $a; $c -= $b; $c ^= ($this->zeroFill($b,5));
   $a -= $b; $a -= $c; $a ^= ($this->zeroFill($c,3));
   $b -= $c; $b -= $a; $b ^= ($a<<10);
   $c -= $a; $c -= $b; $c ^= ($this->zeroFill($b,15));
   return array($a,$b,$c);
 }
 
 function GoogleCH($url, $length=null, $init=GOOGLE_MAGIC) {
  if(is_null($length)) {
   $length = sizeof($url);
  }
  $a = $b = 0x9E3779B9;
  $c = $init;
  $k = 0;
  $len = $length;
  while($len >= 12) {
   $a += ($url[$k+0] +($url[$k+1]<<8) +($url[$k+2]<<16) +($url[$k+3]<<24));
   $b += ($url[$k+4] +($url[$k+5]<<8) +($url[$k+6]<<16) +($url[$k+7]<<24));
   $c += ($url[$k+8] +($url[$k+9]<<8) +($url[$k+10]<<16)+($url[$k+11]<<24));
   $mix = $this->mix($a,$b,$c);
   $a = $mix[0]; $b = $mix[1]; $c = $mix[2];
   $k += 12;
   $len -= 12;
  }
  $c += $length;
  switch($len){
   case 11: $c+=($url[$k+10]<<24);
   case 10: $c+=($url[$k+9]<<16);
   case 9 : $c+=($url[$k+8]<<8);
   /* the first byte of c is reserved for the length */
   case 8 : $b+=($url[$k+7]<<24);
   case 7 : $b+=($url[$k+6]<<16);
   case 6 : $b+=($url[$k+5]<<8);
   case 5 : $b+=($url[$k+4]);
   case 4 : $a+=($url[$k+3]<<24);
   case 3 : $a+=($url[$k+2]<<16);
   case 2 : $a+=($url[$k+1]<<8);
   case 1 : $a+=($url[$k+0]);
  }
  $mix = $this->mix($a,$b,$c);
 /* report the result */
 return $mix[2];
 }
 
 //converts a string into an array of integers containing the numeric value of the char
 
 function strord($string) {
  for($i=0;$i<strlen($string);$i++) {
   $result[$i] = ord($string{$i});
  }
 return $result;
 }
 
 function printrank($url){
  $ch = "6".$this->GoogleCH($this->strord("info:" . $url));
  
  $fp = fsockopen("www.google.com", 80, $errno, $errstr, 30);
  if (!$fp) {
     echo "$errstr ($errno)<br />\n";
  } else {
     $out = "GET /search?client=navclient-auto&ch=" . $ch .  

"&features=Rank&q=info:" . $url . " HTTP/1.1\r\n" ;
     $out .= "Host: www.google.com\r\n" ;
     $out .= "Connection: Close\r\n\r\n" ; 
     fwrite($fp, $out);
     while (!feof($fp)) {
       $data = fgets($fp, 128);
       $pos = strpos($data, "Rank_");
         if($pos === false){
         }else{
           $pagerank = substr($data, $pos + 9);
           $this->pr_image($pagerank);
         }
     }
     fclose($fp);
  }
 }

//
// Display pagerank image. Create your own or download images I made for this script. 
// If you make your own make sure to call them pr0.gif, pr1.gif, pr2.gif etc.
//

 function pr_image($pagerank){
  if($pagerank == 0){
   $this->pr = "<img src=\"images/pr0.gif\" alt=\"PageRank " .$pagerank. " 

out of 10\">" ;
   }elseif($pagerank == 1){
   $this->pr = "<img src=\"images/pr1.gif\" alt=\"PageRank " .$pagerank. " 

out of 10\">" ;
   }elseif($pagerank == 2){
   $this->pr = "<img src=\"images/pr2.gif\" alt=\"PageRank " .$pagerank. " 

out of 10\">" ;
   }elseif($pagerank == 3){
   $this->pr = "<img src=\"images/pr3.gif\" alt=\"PageRank " .$pagerank. " 

out of 10\">" ;
   }elseif($pagerank == 4){
   $this->pr = "<img src=\"images/pr4.gif\" alt=\"PageRank " .$pagerank. " 

out of 10\">" ;
   }elseif($pagerank == 5){
   $this->pr = "<img src=\"images/pr5.gif\" alt=\"PageRank " .$pagerank. " 

out of 10\">" ;
   }elseif($pagerank == 6){
   $this->pr = "<img src=\"images/pr6.gif\" alt=\"PageRank " .$pagerank. " 

out of 10\">" ;
   }elseif($pagerank == 7){
   $this->pr = "<img src=\"images/pr7.gif\" alt=\"PageRank " .$pagerank. " 

out of 10\">" ;
   }elseif($pagerank == 8){
   $this->pr = "<img src=\"images/pr8.gif\" alt=\"PageRank " .$pagerank. " 

out of 10\">" ;
   }elseif($pagerank == 9){
   $this->pr = "<img src=\"images/pr9.gif\" alt=\"PageRank " .$pagerank. " 

out of 10\">" ;
   }else{
   $this->pr = "<img src=\"images/pr10.gif\" alt=\"PageRank " .$pagerank. 

" out of 10\">" ;
  }
 }
 function get_pr(){
  return $this->pr;
 }
}
?>

Usage

Do following:

   1. Save the code above as pagerank.php.
   2. Download or create your own images to display each rank.
   3. Create a directory "images" containing all page rank images. 
   4. See code below on how to use the class. 

<?php
include("pagerank.php");
$gpr = new pageRank();
$gpr->printrank("http://www.yahoo.com");
//display image
echo $gpr->get_pr();
?>


(my thanks to http://centricle.com/tools/html-entities/ for HTML encoding)

Create and send CSV files from PHP

Here's how CSV files are created and downloaded on this site. No saving the file or data import into Excel ... Excel just opens with the data automatically. Very handy.

This function as shown pulls all the field names to create a CSV header and then pulls every field from every row in the table. There is no need to know the field names, the data types or the size of the table. Quotes in the data are double-quoted and the result is surrounded by quotes.

CSV calls for each field to be contained in double quotes.


Pull data from a database using standard PHP MySQL functions:

<?php

$db_link        = mysql_connect(DB_HOST, DB_USER, DB_PSWD);
$db_selected    = mysql_select_db(DB_DATABASE, $db_link);

# Create the CSV file header from the database-table field names
$query          = "SHOW COLUMNS FROM table";
$result         = mysql_query($query);

$csv_output     = NULL;
while ($row = mysql_fetch_assoc($result))
	{
	$csv_output .= '"' . str_replace('"', '""', $row["Field"]) . '",';
	}
$csv_output  = substr($csv_output, 0, -1) . "\n";  // remove trailing "," and add a line break

# Pull all the rows
$query          = "SELECT * FROM table";
$result         = mysql_query($query);

# loop through database records creating one comma-delimed line per row
while ($row = mysql_fetch_assoc($result))
	{
	foreach ($row as $key => $value)
	  {
	  $$key = $value;
	  $$key = str_replace('"', '""', $$key);
	  $var  = $$key;
	  $csv_output .= "\"$var\",";
	  }
	$csv_output .= "\n";
	}

# send the file
$size_in_bytes		= strlen($csv_output);
$csv_file		= "filename_" . date("Y-m-d") . ".csv";
	
$ContentType		= "Content-type: application/vnd.ms-excel";
$ContentLength		= "Content-Length: $size_in_bytes";
$ContentDisposition	= "Content-Disposition: attachment; filename=\"$csv_file\"";

header($ContentType);
header($ContentLength);
header($ContentDisposition);

echo "$csv_output"; 

?>

I generally use compression with output buffering to speed things up:

ob_start("ob_gzhandler");
      .
      .
      .
ob_end_flush();

To use in your HTML:

<button onclick="window.location='CSVoutput.php'" 
	style="font-size:85%;width:100px;">Download<br />CSV file</button>

(my thanks to http://centricle.com/tools/html-entities/ for HTML encoding)

PHP out of memory condition

Fatal error: Allowed memory size of 18388608 bytes exhausted 
(tried to allocate 724 bytes) in /home/dir1/dir2/script.php on line ###

When working with large amounts of data in memory (very long concatenated strings and/or very large arrays in my case), the server may hit a memory max. No amount of "unset" of variables will do the trick past a certain point.

This is the result of a memory allocation ceiling set in php.ini that can be over-ridden (judiciously) as follows:

ini_set  ("memory_limit", -1  );

Be sure to test your code with smaller amounts of data first: this limit is set for a reason ... programs have been known to have been poorly written (not yours, of course; but test anyway).

Ternary Operator and the IIF function

The standard PHP If statement can be reduced by the ternary operator, which is described in the PHP manual Comparison Operators. The IIF function puts the ternary operator into a function.

Ternary Operator

conditional ? if_true : if_not_true;

is the same as

if (conditional)
  {
  if_true
  }
  else
    {
    if_not_true
    }

IIF Function

To return the result of the ternary operator

function iif($expression, $returntrue, $returnfalse = '') {
    return ($expression ? $returntrue : $returnfalse);
}

MySQL server has gone away

MySQL timeout? Probably a new error after years of working perfectly, resulting from an ISP change which they will neither acknowledge nor fix. Sound familiar?

Charming people.

Try this:

$db_link = @mysql_connect(DB_HOST, DB_USER, DB_PSWD,'',MYSQL_CLIENT_INTERACTIVE);

... instead of what you used to do:

$db_link = @mysql_connect(DB_HOST, DB_USER, DB_PSWD);

Images

The panel below shows every image (2000+) in every blog (800+) on Philadelphia Reflections starting with most recent additions.

It works better on some browsers especially Firefox than others, and -- with 2000 images -- it takes a while to load, as much at 10 minutes on a slow connection. An icon in the corner of the picture-wall starts a slideshow. Note: Mouse-clicking enlarges each thumbnail picture, displaying an icon linked to the website source page. We suggest you try out every little icon to see the amazing versatility of Cooliris.

Get Adobe Flash

WRTI, Classical Music and Jazz

{Susan Lewis}
Susan Lewis

Susan Lewis recently entertained the Right Angle Club with a description of her life as the script writer for WRTI, the local classical music station. WRTI could be described as one of three local affiliates of National Public Radio, the network content provider headquartered in Washington DC. The other two are WHYY, a talk station, and WXPN, the University of Pennsylvania station devoted to folk, rock, blues and root music. Another way of describing WRTI is that it took over the role formerly served by WFLN before it was sold, incorporating it into Temple University's jazz station. It plays classical music from 6AM to 6PM, and then plays jazz in the evening. Philadelphia thus really only has half a classical music station, when most cities who are home to a major orchestra have at least two. It is not clear whether this anomaly is a comment on the local radio climate, or the future of its musical one.

{Philadelphia Opera House}
Philadelphia Opera House

The question came up as to just what is classical music, since there are turf boundaries for the affiliates of National Public Radio. Susan Lewis, who has the surprising background of being a former corporate lawyer has apparently given this some thought. She offers the opinion that classical music overwhelmingly consists of music with multiple performers. Orchestras, opera, chorales, and chamber music characterize the topic more than pre-contemporary origins. A brand new symphony would naturally fall into the classical music category, while songs by Frank Sinatra would not, even though excited announcers might call his songs classics. Following this theme, classical music seems to fit with jazz, which consists of several soloists working on variants of a common theme. The sad question thus comes up whether Philadelphia's declining interest in classical music might in some way reflect social fragmentation within a metropolitan community which historically has highly valued cooperation and consensus. One hopes that's not the case.

{Privateers}
WRTI RADIO

Playing a succession of recorded musical selections sounds as though it would be a low-budget operation, but WRTI costs $3.6 million to run, annually. The script writer gets up early, reads the day's artistic news and events, and some auto traffic reports, and records one-minute vocal interludes between the pieces of music. About once a week, a special seven-minute segment is assembled from exerpts from interviews or interludes relating to a theme in the artistic world. One taped recording of carillon music and commentary proved to be quite charming and entertaining, including the news that the carillon in Holy Trinity Church is the oldest in America. Since a bell is a variant of a tuning fork, the bells of a carillon chime with a very long period of decay, creating a problem for both composer and performer to avoid successive notes which conflict unless there is a long pause. These magazine-like pieces of hers are always organized around a main emotional "hook" of some sort, and Susan finds they are very time-consuming to assemble. That leads to a constant succession of inflexible deadlines, just like lawyers' briefs before a legal deadline, generating an excitement strangely exhilarating to the participant, and highly mystifying to outsiders.

As the central focus for dozens of emails and text messages about the goings-on of the local artistic world, the job of town gossip for the art world is an ego trip only suitable for a person who revels, with affection, in the endless wealth of art and anecdote in Philadelphia. As bloggers also know, this job constantly surfaces interesting news tidbits that surprise and please many people. Like the fact that William Penn's hat on top of City Hall is filled with graffiti. Or that a secret colony of Lenni Lanape Indians still exists in town. Or that the forthcoming HP radio standard produces outstandingly high quality.

Ms. Lewis is an asset to our town.

www.Philadelphia-Reflections.com/blog/1521.htm

Forbidden SNL Clip



NBC pulled the original of this Saturday Night Live video from their Web site and replaced it with this edited version.

NBC deleted the section in which Herbert and Marion Sandler described swindling their clients and ultimately Wachovia. The original video had a caption that described the couple as "People who should be shot." Furthermore, the actor portraying Herbert Sandler said "And thank you Congressman Frank as well as many Republicans for helping block Congressional oversight of our corrupt activity."

Herbert and Marion Sandler sold Golden West Financial Corp to Wachovia in 2006 for $24 billion, precipitating Wachovia's failure and eventual sale to Wells Fargo.

Embed Flash as Valid XHTML

The problem to be solved is that you want to embed YouTube (or other Flash movies) but the <embed> tag is deprecated in XHTML and the <param> tags don't validate, either. Here are the steps to clean things up:

The original HTML from YouTube

<object
  width="425"
  height="344">
<param
  name="movie"
  value="http://www.youtube.com/v/zhWkTiMVWVI&color1=0xb1b1b1&color2=0xcfcfcf&hl=en&feature=player_embedded&fs=1">
</param>
<param
  name="allowFullScreen"
  value="true">
</param>
<param
  name="allowscriptaccess"
  value="always">
</param>
<embed
  src="http://www.youtube.com/v/zhWkTiMVWVI&color1=0xb1b1b1&color2=0xcfcfcf&hl=en&feature=player_embedded&fs=1"
  type="application/x-shockwave-flash"
  allowfullscreen="true"
  width="425"
  height="344">
</embed>
</object>

Step 1: replace the closing "</param>" tags with trailing " />"

<object
  width="425"
  height="344">
<param
  name="movie"
  value="http://www.youtube.com/v/zhWkTiMVWVI&color1=0xb1b1b1&color2=0xcfcfcf&hl=en&feature=player_embedded&fs=1"  />
<param
  name="allowFullScreen"
  value="true"  />
<param
  name="allowscriptaccess"
  value="always"  />
<embed
  src="http://www.youtube.com/v/zhWkTiMVWVI&color1=0xb1b1b1&color2=0xcfcfcf&hl=en&feature=player_embedded&fs=1"
  type="application/x-shockwave-flash"
  allowfullscreen="true"
  width="425"
  height="344">
</embed>
</object>

Step 2: put type="application/x-shockwave-flash" into the <object> tag:

<object
  width="425"
  height="344"
  type="application/x-shockwave-flash">
<param
 name="movie"
  value="http://www.youtube.com/v/zhWkTiMVWVI&color1=0xb1b1b1&color2=0xcfcfcf&hl=en&feature=player_embedded&fs=1" />
<param
  name="allowFullScreen"
  value="true" />
<param
  name="allowscriptaccess"
  value="always" />
<embed
  src="http://www.youtube.com/v/zhWkTiMVWVI&color1=0xb1b1b1&color2=0xcfcfcf&hl=en&feature=player_embedded&fs=1"
  type="application/x-shockwave-flash"
  allowfullscreen="true"
  width="425"
  height="344">
</embed>
</object>

Step 3: move src="..." from the <embed> tag to a data="..." attribute in the <object> tag:

<object
  width="425"
  height="344"
  type="application/x-shockwave-flash"
  data="http://www.youtube.com/v/zhWkTiMVWVI&color1=0xb1b1b1&color2=0xcfcfcf&hl=en&feature=player_embedded&fs=1">
<param
  name="movie"
  value="http://www.youtube.com/v/zhWkTiMVWVI&color1=0xb1b1b1&color2=0xcfcfcf&hl=en&feature=player_embedded&fs=1" />
<param
  name="allowFullScreen"
  value="true" />
<param
  name="allowscriptaccess"
  value="always" />
<embed
  src="http://www.youtube.com/v/zhWkTiMVWVI&color1=0xb1b1b1&color2=0xcfcfcf&hl=en&feature=player_embedded&fs=1"
  type="application/x-shockwave-flash"
  allowfullscreen="true"
  width="425"
  height="344">
</embed>
</object>

Step 4: remove the <embed> tag:

<object
  width="425"
  height="344"
  type="application/x-shockwave-flash"
  data="http://www.youtube.com/v/zhWkTiMVWVI&amp;hl=en&amp;fs=1">
<param
  name="movie"
  value="http://www.youtube.com/v/zhWkTiMVWVI&amp;hl=en&amp;fs=1" />
<param
  name="allowFullScreen"
  value="true" />
<param
  name="allowscriptaccess"
  value="always" />
</object>


You can View > Source to see that the code shown here does actually produce the YouTube video displayed.


Step X: it should be noted that in Firefox you don't need any "<param>" tags at all, which makes things very simple and clean:

<object
  width="425"
  height="344"
  type="application/x-shockwave-flash"
  data="http://www.youtube.com/v/zhWkTiMVWVI&amp;hl=en&amp;fs=1">
</object>

Not in IE, though; nope. (Why the </object> instead of a closing " />"? Because it seems to work more reliably in Firefox 3.0.5; I don't know why.)

Book Proposal: Investing

BOOK PROPOSAL
March 3, 2009

Subject/Market
Investing, individual and institutional

Description
Individual and institutional investors have been badly burned in the last decade. Many are on the verge of ruin because of poor investment practices based on bad advice from Wall Street and lax regulatory oversight. This book explains what went wrong and how to invest in the future to ensure a reliable cash flow.

The book is finished; the Contents and Chapter Outline are included here; (161 pages in Microsoft Word using 12pt Times New Roman font, single-spaced; 40,000 words, 50 images, 22 tables). Title: "Investing for Cash Flow". SubTitle: "Individual and institutional investors must change their investment style to survive financial crises".

Author
Economist and Certified Financial Planner™ who has spent 30 years on Wall Street … Managing Director at Morgan Stanley; EVP and CIO at Fidelity Investments; on the Board of Prudential Securities responsible for their merger with Wachovia.

George R. Fisher
5 Pier 7
Charlestown, MA 02129-4225

Cell 917-514-8204
Fax 215-689-4880
email george@georgefisher.com

Similar books

Unconventional Success David F. Swensen
Common Sense on Mutual Funds John C. Bogle
Asset Allocation Roger C. Gibson
The Four Pillars of Investing William Bernstein
Winning Investment Strategy Larry E. Swedroe
The Intelligent Investor Benjamin Graham

Why is this book different?
Many books on investing are filled with the mathematics of Modern Portfolio Theory or recount the history of the markets, and most give general advice; but none actually tell investors what they should do in practice. This book provides a step by step process investors can and should follow to produce a reliable cash flow; the book uses academic analysis and detailed statistics/charts but it is accessible to all experienced investors.

Portions have been posted on the web and have received a very favorable response.

Contents

  • Introduction
    The purpose of investing is to produce a reliable cash flow.
  • Modern Portfolio Theory (MPT)
    A primer on the basics of portfolio construction.
  • Process
    The steps to take

1.      Choose the asset classes

2.      Active management or indexing?

3.      Specific security, closed-end, mutual fund, ETF?

4.      Pick the specific investments

5.      Choose the asset allocation

6.      Manage the investment portfolio using Portfolio Rebalancing

7.      Produce a Reliable Cash Flow

8.      Become a Lobbyist

  • Appendixes
  • Geometric vs. Simple Average
  • The Cost of Costs
  • What good are bond funds?
  • Writing Options as an Alternative to Market Orders
  • Protection of Assets: SIPC and Excess Coverage
  • 1975: Landmark Year
     
  • Notes For Individual Investors
  • Choose the brokerage firm
  • Set account options
  • Taxable and retirement accounts
  • How much can you withdraw from a portfolio?
  • Why not buy an annuity?
  • Effective Tax Rate
  • Rollovers: Why Not & How To
  • Estate Planning: a call to action
  • Time Value of Money
  • Bibliography
  • Disclaimers
  • Index

 

Chapter Outline

  • Introduction
    The purpose of Investing is to produce a reliable cash flow. Of the thousands of books and conferences on Investing not one reflects on this vital fact: every investor needs to produce a reliable cash flow. This applies to charitable foundations, university endowments and municipal pension funds just as much as it does to individuals. While chasing yields during bull markets, most investors forget that come the inevitable downturn they will still be required to cover their expenses.

    Starting in 2007, this lesson was brought brutally home (again) to the whole world. Every single major financial institution failed its clients and the regulators were entirely absent. Successful Investing seems like a bad joke during such a time but it always does when the market turns down.

    Investors need to take an entirely new approach going forward to establish their independence from the financial services industry and to focus on producing the one thing that ultimately matters: reliable cash flow.
  • Modern Portfolio Theory (MPT)
    A primer on the basics of portfolio construction. MPT teaches us to diversify widely, to hold uncorrelated assets and to keep our eyes on the long term rather than the momentary gyrations of the markets. The theoretical work was done in the 1950s and there is recent empirical evidence that it will work to every investor's benefit if sensibly and conservatively applied.
  • Process
    The steps to take: there is a logical sequence of steps every investor must take to construct a portfolio that will produce a reliable cash flow year in and year out. The basics are drawn from Modern Portfolio Theory and avoid any reliance on the financial services industry. There is nothing revolutionary in all of this, except for the fact that very few investors have actually followed these precepts. Much of this process involves following the advice of Benjamin Graham in 1939, updated for modern circumstances & products and with the benefit of 70 more years of experience and academic study upon which to draw.
  1. Choose the asset classes
    You must decide what you will own, what you will invest in. Equity, debt, real estate, commodities and cash cover most of the viable options pretty comprehensively. We need to look at the pros and cons of each asset class, both in the US and internationally.
     
  2. Active management or indexing?
    Will you try to beat the market or "settle" for just being average? It turns out that just being average will consistently beat anything offered by the financial services industry which always promises to do better but never does.
  3. Specific security, closed-end, mutual fund, ETF?
    What "vehicle" should you choose to invest in? Index Viper and iShare ETFs will provide the best results for your investment portfolio; for your cash portfolio, you need to look somewhat further afield.
  4. Pick the specific investments
    You've decided on the asset classes and you've decided on the investment vehicles, which specific securities are best suited, and how many positions should you hold?
  5. Choose the asset allocation
    Beebower, Brinson and Hood's famous 1986 study convinced most of the world that fine-tuning the exact asset allocation of a portfolio would have a significant effect on returns. It turns out that not only does it not do this, but the study wasn't even about returns in the first place. Asset allocation is about risk, not return.

    A variety of current portfolios are presented to provide perspective.
  6. Manage the investment portfolio using Portfolio Rebalancing
    Portfolio Rebalancing is one of the most vital and the most misunderstood techniques in all of Investing. Done correctly, portfolio rebalancing holds the key to capturing investment gains as well as providing the most tax-efficient way to produce cash, which is (after all) the purpose of all this effort.
  7. Produce a Reliable Cash Flow
    The purpose of any investment is to produce a reliable cash flow. That cash flow may not be needed until sometime in the future; or the operating needs of an organization may depend upon it for daily functioning. In any event, a portfolio must consist of two parts: investment and cash; and the establishment and maintenance of the cash portfolio is the key to long-term investment success.
  8. Become a Lobbyist
    Capitalism may not be dead but it has become anesthetized recently. For over a decade the risk-free US Treasury has produced a better total return than any equity market. The risk/return tradeoff of MPT has not been working.

    This is upside down and it is the result of the twin failures of the financial services industry and Governmental oversight. If equity Investing does not start producing a better return than Treasuries, what rational person would invest? Investors must insist that certain basic principles be enforced going forward or Investing will wither.
  • Appendixes
  • Geometric vs. Simple Average
    How are investment returns measured?
  • The Cost of Costs
    The financial services industry's primary purpose is to generate fees; an investor's primary focus must be to eliminate them. Costs eat up an average of 50% of an investor's returns ... for absolutely nothing.
  • What good are bond funds?
    Why hold a bond fund instead of a bond portfolio? The answer has to do with the inner workings of the yield-to-maturity calculation.
  • Writing Options as an Alternative to Market Orders
    Portfolio Rebalancing offers investors the opportunity to increase their returns slightly by writing options instead of executing market (or limit) orders. Not for everyone, but a viable alternative for some investors once their portfolios are established correctly.
  • Protection of Assets: SIPC and Excess Coverage
    The collapse of several titans has focused attention on various forms of asset insurance. The Government's guarantee is probably pretty good (if you're not in a hurry) but private, "excess" insurance coverage is not. Diversify among institutions as well as among securities.
  • 1975: Landmark Year
    Most investors don't know it but May 1, 1975 marked their Independence Day … if only they would take advantage of their freedom.
     
  • Notes For Individual Investors
    Most of the advice of this book applies equally to institutional as well as individual investors. Individuals do have some special requirements, however.
  • Choose the brokerage firm
    Never put your money in a "full service" brokerage account. Never put your money in a mutual fund or insurance account. Never turn your money over to someone else to manage. It may seem déclassé or unsophisticated to use a "discount broker" but such pride is badly misplaced. Which ones are best?
  • Set account options
    There are several options that should be set for your brokerage account for the best results. Set them upon establishment if possible.
  • Taxable and retirement accounts
    "Asset location" refers to the problem of deciding what to put into a taxable account or a retirement account. The answer has to do primarily with taxes (which change from time to time) and cash-withdrawal needs.
  • How much can you withdraw from a portfolio?
    It is vitally important that you not run out of money once you are living off your investments. Based on 30-years' history, studies have concluded that 4% adjusted for inflation is the maximum prudent withdrawal rate for a portfolio diversified among equity and debt.
  • Why not buy an annuity?
    An annuity can provide a level of peace of mind and as such a well-chosen immediate annuity can be a valuable component of a retiree's portfolio. But understand the tradeoffs before you run out and buy one.
  • Effective Tax Rate
    The IRS wants everything it's owed but the rules require of you no more than the minimum. Don't be unintentionally generous.
  • Rollovers: Why Not & How To
    The financial services industry is desperate to get you to roll over your retirement plan into an IRA. This is not always in your best interest.
  • Estate Planning: a call to action
    Little mentioned except in political campaigns, estate taxes are the most onerous in the whole tax code. If you think a simple will can protect you, think again.
  • Time Value of Money
    If you don't understand the basics of compound interest, present value and so on, you should take the trouble to learn.
book Proposal: Investing for Cash Flow

CNBC Exposed

This series of videos is the best insight into America's financial reporting you will ever find.












Here's the weasel talking to another weasel



Macroeconomics of The 2007 Collapse

Sudden wealth creation, whether from the discovery of gold or oil, the conversion of poverty into useful cheap labor, or the sudden abundance of cheap credit, is of course a good thing. Sudden wealth creation can be compared with a stone thrown into a pond, causing a splash, and ripples, but leaving a somewhat higher water level after things calm down. The globalization of trade and finance in the past fifty years has caused 150 such disturbances, mostly confined to a primative developing country and its neighbors. Only the 2007 disruption has been large enough to upset the biggest economies. It remains to be seen whether disorder to the whole world will result in revised world monetary arrangement. One hopes so, but national currencies, tightly controlled by local governments, have been successful in the past in confining the damage. This time, the challenge is to breach the dykes somewhat, without letting destructive tidal waves sweep past them. Many will resist this idea, claiming instead it would be better to have higher dykes.

It is the suddenness of new wealth creation in a particular region which upsets existing currency arrangements. Large economies "float" their currencies in response to the fluxes of trade, smaller economies can be permitted to "peg" their currencies to larger ones, with only infrequent readjustments. Even the floating nations "cheat" a little, in response to the political needs of the governing party, or to stimulate or depress their economies as locally thought best. All politicians in all countries therefore fear a strictly honest floating system, and their negotiations about revising the present system will surely be guilty of finding loopholes for each other; the search for flexible floating will therefore claim to seek an arrangement which is "workable".

In thousands of years of governments, they have invariably sought ways to substitute inflated currency for unpopular taxes. The heart of any international payment system is to find ways to resist local inflation strategems. Aside from using gunboats, only two methods have proven successful. The most time-honored is to link currencies to gold or other precious substances, which has the main handicap of inflexibility in response to economic fluctuations. After breaking the link to gold in 1971, central banks regulated the supply of national currency in response to national inflation, so-called "inflation targeting". It worked far better than many feared, apparently allowing twenty years without a recession. It remains to be investigated whether the substitution of foreign currency defeated the system, and therefore whether the system can be repaired by improving the precision of universal floating, or tightening the obedience to targets, or both. These mildest of measures involve a certain surrender of national sovereignty; stronger methods would require even more draconian external force. The worse it gets, the more likely it could be enforced only by military threat. Even the Roman Empire required gold and precious metals to enforce a world currency. The use of the International Monetary Fund (IMF) implies attempts to dominate the politics of the IMF. So it comes to the same thing: this crisis will have to get a lot worse, maybe with some rioting and revolutions, before we can expect anything more satisfactory than a rickety negotiated international arrangement, riddled with embarassing "earmarks". Economic recovery will be slow and gradual, unless this arrangement is better, or social upheavals worse, than would presently appear likely.

Sold Out

Valid XHTML YouTube embed code generator

Date Math

Figuring out how many days there are between two dates is either built in to the system you are using or else you will tear your hair out what with leap years, floating leap days, etc.

There are two ways to do this fairly simply:

  1. Using Julian Date Counts
  2. Using the system clock functions

Julian Date Counts


Julian Date Counts are not related to Julius Caesar's calendar. They are a way to assign a sequential number to every date going way back into antiquity. Convert any two dates to their Julian Date Count number and the difference is the number of days between the dates.

See Julian Day Numbers

In C++ ...

long int GregorianToJulian(int gregorianMonth, int gregorianDay, int gregorianYear)
	{
	// this function calculates the Julian Date Number from a Gregorian date
	//
	// astronomers and others use these numbers becasue the difference between two
	// Julian Date Numbers is the number of days between the dates
	//
	// see http://quasar.as.utexas.edu/BillInfo/JulianDatesG.html

	long int A;
	long int B;
	long int C;
	long int E;
	long int F;
	long int JD;

	long int Y, M, D;

	Y = gregorianYear;
	M = gregorianMonth;
	D = gregorianDay;

	if (M <= 2)
	  {
	  Y = Y - 1;
	  M = M + 12;
	  }
	
	A = Y/100;
	B = A/4;
	C = 2-A+B;
	E = 365.25*(Y+4716);
	F = 30.6001*(M+1);
	JD= C+D+E+F-1524.5;

	return JD;
	}


void JulianToGregorian (long int julianDateNumber, int& gregorianMonth, int& gregorianDay, int& gregorianYear)
	{
	// Convert a Julian Date Number to a Gregorian Date
	//
	// see http://quasar.as.utexas.edu/BillInfo/JulianDatesG.html

	long int Z;
	long int W;
	long int X;
	long int A;
	long int B;
	long int C;
	long int D;
	long int E;
	long int F;

	Z = julianDateNumber+0.5;
	W = (Z - 1867216.25)/36524.25;
	X = W/4;
	A = Z+1+W-X;
	B = A+1524;
	C = (B-122.1)/365.25;
	D = 365.25*C;
	E = (B-D)/30.6001;
	F = 30.6001*E;

	gregorianDay   = B-D-F;
	gregorianMonth = ((E-1) > 12) ? E-13 : E-1;
	gregorianYear  = (gregorianMonth>2) ? C-4716 : C-4715;
	
	return;
	}


Using the system clock functions


Again in C++ ...

/*

Find the days between two dates

Found at 
http://stackoverflow.com/questions/1381832/how-to-calculate-the-number-of-days-between-two-given-dates-leap-year-obstacle
and http://www.cplusplus.com/reference/clibrary/ctime/mktime/

The sources were helpful but I had to extensively modify/hack to make this actually work

*/

#include <time.h>
#define SECONDS_PER_DAY (24 * 60 * 60)

time_t time_from_date(int year, unsigned int month, unsigned int day)
{
    // Return the time from the year 1900 to the date entered
	
    struct tm a = {0,0,0,day,month - 1,year - 1900};
    time_t x = mktime(&a);

    return x;
}

int days_between(int year0, unsigned month0, unsigned day0,
                 int year1, unsigned month1, unsigned day1)
{
	// The difference in seconds between two dates,
	//   divided by the number of seconds in a day ...
	//
	//     is the number of days between the dates

    return difftime(time_from_date(year1, month1, day1),
                    time_from_date(year0, month0, day0)) / SECONDS_PER_DAY;
}

(my thanks to http://centricle.com/tools/html-entities/ for HTML encoding)

Escaping for PHP Output to JavaScript

To send data to a JavaScript script from PHP, three levels of escaping are required as shown in the snippet below:

<script type="text/javascript">
// <![CDATA[


// escape for JavaScript
$message   = preg_replace("/\r?\n/", "\\n", addslashes($message));

// escape for XHTML
$message   = preg_replace('%</%i', '<\/', $message);

// send to JavaScript
echo "   var message = \"$message\";\n";


// ]]>
</script>

(my thanks to http://centricle.com/tools/html-entities/ for HTML encoding)

Google Maps Icons

Google Maps/Earth do not make icon creation & manipulation easy. Here are a couple of tips:

GIcon (look here: http://code.google.com/apis/maps/documentation/reference.html#GIcon) has a number of methods for creating and modifying an icon. I've found it's best to start with the default because adding features you expect is harder than you think.

// Here's how to create a new icon with the defaults
var newIcon = new GIcon(G_DEFAULT_ICON);

// To create a new icon like the default but yellow:
var yellowIcon = new GIcon(G_DEFAULT_ICON, "http://www.philadelphia-reflections.com/images/googlemapsmarkeryellow.png");

// To make use of that yellow icon and give it a tooltip
var point   = new GLatLng(40.39, -75.34);
var marker  = new GMarker(point, {icon:yellowIcon, title:"View Above Philadelphia"});
map.addOverlay(marker);

// To open a balloon when clicked
var message = " ... fill with text and HTML ... I've found tables are very helpful ";
GEvent.addListener(marker, 'click', function() {marker.openInfoWindowHtml(message);});

// Change icon on mouseover (see http://www.cems.uwe.ac.uk/~cjwallac/apps/phpxml/showIcons.php)
var msoverIcon = new GIcon(G_DEFAULT_ICON, "http://maps.google.com/mapfiles/kml/pal2/icon1.png");
GEvent.addListener(marker, 'mouseover', function() { marker.seticon(msoverIcon); });

Note: the "message" part of that snippet has to be escaped correctly.
See http://www.philadelphia-reflections.com/blog/1783.htm


(my thanks to http://centricle.com/tools/html-entities/ for HTML encoding)

Existing Websites Which Offer Higher Education

Immense: Here are a few of my favorites--

http://www.khanacademy.org/

http://academicearth.org/

http://videolectures.net/

http://ocw.mit.edu/OcwWeb/web/home/home/index.htm

http://mitworld.mit.edu/browse

http://www.youtube.com/user/MIT

http://watch.mit.edu/

http://numericalmethods.eng.usf.edu/

http://www.apple.com/education/itunes-u/whats-on.html

Unemployment 2008/09

QR Codes

QR Codes are similar to bar codes in that they are read optically. Most-common in Japan, all Japanese cell phones can read them; all fancy phones in America (iPhone, etc.) have download-able apps that can read QR Codes (semacode is a free QR Code app for the iPhone but there are many for all).

One application that is becoming common is encoding a website's URL and including the image in a print advertisement.

The QR Codes below were created by http://qrcode.kaywa.com/; create QR and semacode/DataMatrix from text: http://invx.com/code/.

QR Codes
{philadelphia reflections qrcode} {george fisher qr code}
Philadelphia Reflections George Fisher (Flash... N/G on iPhone)
{chemical heritage society qr code} {kaiser qr code}
Chemical Heritage Society Kaiser Permanente
{george fisher advisors qr code} http://www.philadelphia-reflections.com/images/missing_img.gif
George Fisher Advisors QR Code George Fisher Advisors semacode/DataMatrix

Find your location

Click the button at the top of your browser to allow it to share your location.

Your browser does not support iframes.

Here's the key Javascript/HTML code (thanks to http://maxheapsize.com):

<head>

<script type="text/javascript">
// <![CDATA[

var map;
var geocoder;

function initialize() {
   map = new GMap2(document.getElementById("map_canvas"));
   map.setCenter(new GLatLng(34, 0), 1);
   geocoder = new GClientGeocoder();
   }

if (navigator.geolocation) {
  
   navigator.geolocation.getCurrentPosition(function(position) {  
	var address = position.coords.latitude+","+position.coords.longitude;
	geocoder.getLocations(address, addAddressToMap);
 }); 
  
} else {
  alert("I'm sorry, but geolocation services are not supported by your browser.");
}  

function addAddressToMap(response) {
      map.clearOverlays();
      if (!response || response.Status.code != 200) {
        alert("Sorry, we were unable to geocode that address");
      } else {
        place = response.Placemark[0];
        point = new GLatLng(place.Point.coordinates[1],
                            place.Point.coordinates[0]);
        marker = new GMarker(point);
        map.setCenter(point, 13);
        map.addOverlay(marker);
		var addr = document.getElementById('address');
		addr.firstChild.data = place.address;
      }
    }

// ]]>
</script>

</head>

<body onload="initialize()">

<div id="map_canvas" style="width: 500px; height: 300px"></div>

Python URL Handling

In case you're wondering "How the heck does Python handle headers and data under 3.2.2?", here's an example that works using IDLE and Python 3.2.2 installed on a 64-bit Windows 7 machine.

import re
import urllib.request
url = "http://www.philadelphia-reflections.com"

uf = urllib.request.urlopen(url)

# header information
print('--- headers ---')
info = uf.info()  # headers

#headers = info._headers # a list of all the headers

print('charsets:',info.get_charsets())
print('content_charset:',info.get_content_charset()) 
print('content_type:',info.get_content_type())
print('content_maintype:',info.get_content_maintype())
print('content_subtype:',info.get_content_subtype())
print('default_type:',info.get_default_type())
print('filename:',info.get_filename())
print('params:',info.get_params())
print('payload:',info.get_payload())

print()
print('--- data ---')

data = uf.read().decode(info.get_content_charset()) # content
print(data[:500])

print()
print('--- image ---')

imageurl = url + "/images/001.JPG"
image = urllib.request.urlretrieve(imageurl, 'python_001.jpg')
print(image)

(my thanks to http://centricle.com/tools/html-entities/ for HTML encoding)

Website Test Results

The site www.webpagetest.org is an excellent facility for testing the performance of a web site. Philadelphia Reflections passes with flying colors:

{Response time test with lots of images}
Response time test with lots of images

REFERENCES


Excellent site for website testing www.webpagetest.org

Iterate through a Word document, modifying picture properties(Blog 2300)

(Blog 2300) We have a facility on this website to download books of many chapters (made up of volumes of topics on the site) to Microsoft Word for subsequent editing and eventual publishing. In many cases we download lots of pictures (via an img src= tag). I have not found a way to set the way text flows around the images in Word using HTML or CSS, so I built a Word macro to do it. This should allow you to change the size of images, as well as move them around. Moving the captions requires the use of the captions feature in Word's image menu (right-click).

------------------------------------

Instructions for use of a Macro named Sub ImageFlow():

  1. Open Word

  2. In Word, enter File>Open

  3. enter the URL of the file you want to modify into the File Entry box and press the Enter key to load the document. It may take a minute or two, but a working screen should appear, loaded with the file in a condition ready to move the pictures around.

  4. Press Alt + F11 which will open the VBA screen

  5. Copy the macro found on this page from
    Sub ImageFlow()
    to
    End Sub
  6. In the right-hand panel of the VBA screen press Ctrl+A, Ctrl+V to paste it in

  7. In the VBA screen press F5 to run the macro

If you want to do a lot of these manipulations, save the macro in the Macro Library of Windows Word.

------------------------------------
Sub ImageFlow()
'
'  this Macro goes through an entire Word document and
'  changes the way text flows around each picture
'  ("Tight" in this example but see below for choices)
'
    Dim shpIn As InlineShape, shp As Shape

    For Each shpIn In ActiveDocument.InlineShapes
        If (shpIn.Type = wdInlineShapeLinkedPicture) Then
            Set shp = shpIn.ConvertToShape
            shp.WrapFormat.Type = wdWrapTight
        End If
    Next shpIn

    For Each shp In ActiveDocument.Shapes
        shp.WrapFormat.Type = wdWrapTight
    Next shp

End Sub
----------------------------------------

Change wdWrapTight to any of the following:
wdWrapBehind
wdWrapFront
wdWrapInline
wdWrapNone
wdWrapSquare
wdWrapThrough
wdWrapTight
wdWrapTopBottom

My thanks to http://www.phrebh.com/Jenius/252-center-pictures-in-word-with-vba/ for showing me the essential technique of iterating through the pictures.

What are the InlineShapes' Types? See http://msdn.microsoft.com/en-us/library/microsoft.office.interop.word.inlineshape.type(v=office.11).aspx; it is possible we may also need to select on wdInlineShapePicture (as well as wdInlineShapeLinkedPicture) but for my specific purpose I did not need to.

SMTP Authorization and Handling Bounced Emails with PEAR Mail

Recently our ISP started requiring user signon in order to send emails. PHP's mail function stopped working as a result.

Naturally, the ISP did not notify us of this change so we were quite surprised when many thousands of emails on our newsletter list were rejected (every one of them, in fact).

What error message was returned to us to notify us of what the problem was? Why this helpful note:

Mail sent by user nobody being discarded due to sender restrictions in WHM->Tweak Settings

Doesn't that just say it all?

I'm being snide, but our ISP is really quite good about keeping its software up to date and aside from an occasional surprise like this, they are very reliable. Being up to date included the automatic incorporation of the PEAR Mail facility which we are now using.

PEAR's Mail system works quite well but two problems were very vexing until we stumbled our way to a solution:

  1. How, exactly, do we sign on to the SMTP server?
  2. How do we ensure that bounced emails (the bane of all email lists) get returned to us?

You might not think that the first question would be so hard but it actually took a good deal of trial and error to get it right. As for the second question, there is an awful lot of wrong information available out in Internet land (including but not limited to VERP and XVERP which I advise you to avoid).

With PEAR Mail you first set up a "factory" and then send emails, either singly or in a loop. We keep the user id, password, etc. in a file "above" the web server in hopes that will keep them secret ... here's the code (it actually is in production and it does in fact work):

<?php
include('Mail.php');

# the email constants are contained in a file outside the web server
include("/level1/level2/level3/constants.php");

$headers = array (
         'From' => '"name"<addr@domain.com>',
         'Sender' => '"name"<addr@domain.com>',
         'Reply-To' => '"name"<addr@domain.com>',
         'Return-Path' => 'addr@domain.com',
         'Content-type' => 'text/html; charset=iso-8859-1',
         'X-Mailer' => 'PHP/' . phpversion(),
         'Date' => date("D, j M Y H:i:s O",time()),
         'Content-Language' => 'en-us',
         'MIME-Version' => '1.0'
         );

// call the PEAR mail "factory"
$smtp = Mail::factory('smtp',
      array (
            'host' => EMAIL_HOST,
            'port' => EMAIL_PORT,
            'auth' => true,
            'username' => EMAIL_USERNAME,
            'password' => EMAIL_PASSWORD,
            'persist' => true,
            'debug' => false
            ), '-f addr@domain.com'
      );

# to send emails:
#
# $headers['To']      = $to;        # provide the "$to" variable, something like $to = '"name"<addr@domain.com>';
#                                   # note that the first parameter of $smtp->send can be "decorated" this way or just a naked email address
# $headers['Subject'] = $subject;   # provide the "$subject" variable
# $mail = $smtp->send($to, $headers, $contents_of_the_email);
#                          -------- ................................> except for 'To' and 'Subject',
#                                                                     $headers is provided by this module but can be over-ridden
# if (PEAR::isError($mail))
# {
#   echo "<p style='color:red;'>The email failed; debug information follows:<br />";
#   echo $mail->getDebugInfo() . "<br />";
#   echo $mail->getMessage()   . "</p>";
# }
# else
# {
#   echo "<p>email successfully sent</p>";
# }

?>

My thanks to http://htmlentities.net/ for the HTML entites conversion.

Data Sources for Health Care

OTHER REVENUE PROVIDERS WITH POTENTIALLY USEFUL MEDICAL DATA, MOSTLY UNUSED

Although some research discoveries are stumbled on by accident, most of the important ones derive from asking the right questions. If you don't ask the right question, you can wander around in a laboratory white coat for a lifetime without discovering much that is worth knowing. We already have huge stores of data, much of it in electronic form, about the health system. It mostly comes from people paying bills:

Health Insurance

Health Savings Accounts

Payroll deductions for Medicare

Medicare premiums

Military Medical Systems

Veterans Administration

Government subsidies to Hospitals

Medicaid (50-70% Federal)

Social Security

Life Insurance

Premium Investment Income

Cash payments (weak source)

Unclassified Remainder

To summarize the data sources already in existence raises questions of privacy and overwhelming government intrusion into the lives of citizens. That might well be a threat in forty or fifty years, but the disaster of the Health Insurance Exchanges trying to use a small particle of this data is reassuring, in a discouraging sort of way. These systems were originally devised to ask questions of no great relevance to national health costs, so they pose no great temptation to a wandering medical snooper. But they almost always have to meet some sort of an annual budget, so the answer to the question we are now asking is mostly available to everybody, on the Internet. It should be comparatively easy to learn, with adequate accuracy, how much is being spent on what kind of person, right now. If the total comes anywhere near 18% of GDP, we have as much detail as we need for this book to defend the conclusions it draws. We can tell the gross amounts, and by dividing by 350 million, get the average per person costs. Apportionment by age is somewhat less precise, but the numbers are so large, age stratification can be fairly accurately estimated. Let's start with a question we think we know the answer to.

 

Please Let Us Know What You Think

 
 

(HTML tags provide better formatting)
 

Blogs

XHTML vs. HTML
XHTML is advanced HTML. Not all browsers support it, so pages must test first and serve what's supported.

PHP output buffering improves response time. Gzip speeds internet transmission, compressing text volume (not images) up to 80%.

Webpage Printing
Webpage printing is supported on this site. It seems to work pretty well except for text flow-around for some browsers.

Floating Three-Column CSS Layout
A popular web page layout is three columns with a header and footer. This is achieved on this web site with CSS using floating columns.

Open a new window with XHTML
Prior to XHTML, you could open a new window with a link by saying target="_blank". That's no longer allowed, but what can you do?

DHTML, PHP and MySQL References
What are good references for advanced website development?

Web hosting providers
We have used two service providers: one good, the other poor.

Regular Expressions
Regular Expressions, regex, are an obscure but very powerful pattern-matching tool that every developer should learn.

Javascript: document.write and XHTML
Document.write does not work with "true" XHTML. Don't bother trying to fix the javascript.

Captcha
Captcha is the term for security codes the user must enter into a form

RSS, Atom, Syndication, etc.
The world is full of XML and XML-like file formats for syndication-feed purposes. Why they must all be different, God alone can tell. But the reason there is lousy documentation is the evil work of Man.

Web Standards Validation
It is important to confirm that your website conforms to standards

Ampersand Madness: Convert &#x26; to &#x26;amp; to prevent XHTML errors
A regex solution to the huge problem of ampersand encoding in XHTML

CSS Zen Garden Suggestions
Here's a list of the designs I think are worth a look, illustrating the great power of CSS.

Font Families
A survey of the most-commonly installed fonts found on Windows machines

Emails From Iraq (2)
Iraq and a hard place: email record in 2007 from a young American in Iraq to work as a consultant to a local company

Geo Positioning
With the advent of Google Earth, the tagging of websites, blogs and photographs with latitude and longitude information has taken a great leap forward.

Process .htm and .html as php
How to include php scripts in html files

Call KML files from within a blog or topic
How to create a button to call a KML or KMZ file

Send a KML file from disk using PHP
Preprocessing a kml or kmz disk file improves the user experience

BU to Pier 7 GPS Tour
Example of Including a KML file

Static vs Dynamic URLs
Implementing static URLs for a website driven by PHP and MySQL is as easy as a little regex and htaccess magic.

Round the Block in Haddonfield
Walking the dog in Haddonfield can be complex.

HTML Forms
How to open a form in a new window when a radio button is clicked.

Regex URL Matching
On this site we check for the existence of a URL whenever an entry is updated. A Regex (regular expression) string was the breakthrough.

Parsing name-value pair attributes in an HTML tag
Regexp HTML Attribute Parsing: Pulling out the value of numerous attributes in an HTML tag is a mind bender

Health Expenditures as a percent of GDP
The US leads the world in health expenditures as a percent of GDP and the number is growing. Is this a meaningful statement?

Server-Side gzip Compression
You can include code in a PHP program to compress an outbound web page. It works, but instructing the web server to do it to every page is much easier.

Central Securities
An analysis of Central Securities (CET).

SQL To Exclude A List Of Items
How do you select everything from one table except for a list contained in another table?

The Evolution of a Programmer
Excellence depends upon simplicity

HTML Anchor without an HREF?
How to execute a JavaScript function and nothing else.

Retirement Planning Video
The retirement situation and how much to save for yourself.

Debunking third-world myths with the best stats you've ever seen
Guaranteed: You've never seen data presented like this. Trends come to life. And the big picture snaps into sharp focus.

Income vs Life Expectancy over time
A graph that shows life expectancy increasing over the years as the income of third-world countries grows.

PHP script to display Google PageRank
It is very handy to know the Google PageRank of your pages. Here's a PHP script that figures it out for you.

Create and send CSV files from PHP
The ability to create a CSV file from MySQL data in PHP, download it and have it open automatically in Excel is very handy.

PHP out of memory condition
PHP scripts sometime run out of internal memory. Here's how to get around the problem.

Ternary Operator and the IIF function
Reduce the PHP If function and/or include it in a function

MySQL server has gone away
What to do when your MySQL connection is timing out for no apparently good reason, all of a sudden.

Images
The 1000+ images on Philadelphia Reflections are displayed at this location. It works better with some browsers than others.

WRTI, Classical Music and Jazz
American RevolutionaryFor a city with such a strong musical presence, it is surprising that Philadelphia has only one classical music radio station.

Forbidden SNL Clip
Forbidden SNL Clip

Embed Flash as Valid XHTML
The embed tag doesn't validate with XHTML. Here's what to do.

Book Proposal: Investing
Investing for Cash Flow
Individual and institutional investors must change their investment style to survive financial crises

CNBC Exposed
Finally some straight talk about the preeminent business news channel. From the Daily Show with Jon Stewart..

Macroeconomics of The 2007 Collapse
What happened to America in 2007 has happened to hundreds of developing economies in the past fifty years.

Sold Out
How Wall Street bought Washington ... How Washington sold out to Wall Street.

mysql_insert_assoc
How to make MySQL insertions easier (and safe)

Internet, Websites, and related Programming
Technical Comments related to programming this and other websites require a special topic section of their own.

mysql_update_assoc
Easily (and quote_smart-ly) update a record in a MySQL database table

Real Estate Investment Calculator
The price you should pay for income-producing property is a function of the cash flow. Too many investors look at criteria other than cash flow and end up making bad investment decisions.

Valid XHTML YouTube embed code generator
This free tool will create a valid XHTML embed code for any YouTube video. The code YouTube shows on the embed field is not valid XHTML! However, you can simply use this simple tool to make it Valid XHTML 1.0 Transitional.

Date Math
If the function to calculate the number of days between two dates isn't built in, it's a pain to figure out

Escaping for PHP Output to JavaScript
To send data to a JavaScript script from PHP, three levels of escaping are required

Google Maps Icons
Google Maps/Earth do not make icon creation & manipulation easy.

Existing Websites Which Offer Higher Education
The number of Internet websites which currently offer free education on the college level is immense and growing rapidly. Just as terabyte is the next step after gigabyte, we need a new word to denominate "much larger than merely immense".

Unemployment 2008/09
The average national unemployment rate of 10.6% does not convey the same impact as seeing the spread of it over time. Tune in for a 30-second display of a graphic display.

QR Codes
QR Codes are becoming common in print ads to encode the sponsor's URL

Find your location
Browsers are starting to support features of the new HTML 5 standard ... such as geo location. Not IE, of course, but try Firefox.

Python URL Handling
An example of URL handling using Python 3.2.2 installed on Windows 7

Website Test Results
Response time test with lots of imagesPhiladelphia Reflections has excellent response time

Iterate through a Word document, modifying picture properties(Blog 2300)
Iterate through a Word document modifying modifying the Wrap Text property of every picture

SMTP Authorization and Handling Bounced Emails with PEAR Mail
Sign on to an SMTP server and get bounced emails returned to you

Data Sources for Health Care
REVENUE PROVIDERS WITH POTENTIALLY USEFUL MEDICAL DATA, MOSTLY UNUSED