A ban on missile research after World War I resulted in development of a large German civilian rocketry program to fill gap. Gerhard Zucker, a German businessman, promoted delivery by Rocket Mail in era of high rocket interest. Zucker unsuccessfully launched rockets filled with post, in both, Germany and Britain in 1933-34, according to Encyclopedia Astronautica. Rockets were propelled by little more than fireworks, but the scorched mail from the failures only resulted in greater collector interest! Zucker died in 1985, but returned to rocketry through much of life and a student's death in sixties, resulted in a ban on German civilian rocketry. Another article at Cabinet magazine provides a Zucker bio and audio of an actual Zucker rocket mail launch, here. Is it just interesting theater by a con man?
There were many attempts by others to deliver mail faster. Willie Ley, a German space pioneer, supervised operation of two rocket mail planes in 1936 at Greenwood Lake, NY. A You Tube video of rocket planes is available, but appears lackluster effort. In 1959, USS Barbero submarine launched Regulus cruise missile with 3,000 commemorative pieces of "Missile Mail" from Norfolk, VA hitting its target 22 min. later - the Naval Auxillary Air Station in Mayport, Florida in a demonstration of US missile might!
Wikipedia entry includes "amateurs" who entered postal space race in mid-1950s, "...'amateur' rocketeers flew a number of zinc dust/sulfur 'micrograin' solid propellant mail-carrying rockets interstate, from California, across the Colorado River, and into Arizona. The postal covers were printed for each occasion, and franked at the nearest destination post office." Technologist Robert Zubrin, of Mars Society fame, still thinks rocket mail could be commercial success with reusable launch systems and ability to deliver anywhere in world in 30-45 minutes. Please, the US Postal Service isn't delivering any faster!