Thursday, June 01, 2006

Today is my last day at Xceed... snif...

I leave lighthearted, because I have made sure the transition here goes smootly. I have completed AES encryption support in Xceed Zip for .NET, which was released a few weeks ago, and helped Jacques complete HTTP proxy support in Xceed FTP for .NET, currently in RC tests for the next package.

It's been an incredible 9 years, and I thank Odi and Daniel for given me the latitude to come up with ambitious and non-conventional designs like the FileSystem Core.

I also feel very lucky to have worked with such talented developers and coworkers. You are a second family. I'm sure the rest of the team won't be jealous if I address special thanks and admiration to Pascal, with whom I have designed and developed Xceed Zip v4+, Xceed Backup, Xceed Winsock, Xceed FTP and the amasing experience and fun we had creating Xceed Zip for .NET.

Finally, It's been an honor having the chance to talk and chat with a few customers, through support, forums and this blog. It is very important for software developers building commercial applications to stay in touch with the customers, instead of hiding behind the marketing or support staff. True input comes from customers.

Thank you.

6/1/2006 8:20:06 AM (Eastern Daylight Time, UTC-04:00)  #   
 Wednesday, April 26, 2006

Is it just me, or that new name for MSH (aka Monad), the new managed MS Shell, sounds aweful? PowerShell? Argh...

4/26/2006 7:03:32 AM (Eastern Daylight Time, UTC-04:00)  #   
 Wednesday, April 05, 2006

It's been a long time since I've posted something "chewable". My energies were all directed on the soon to be released Xceed Zip for .NET version 3.0. Though the apparent changes on the public interface are quite minor, and one can teach the new classes quite easily, the underlying code wasn't trivial.

In short, for all of you who know what ZipArchive, ZippedFile and ZippedFolder are, say hello to TarArchive, TarredFile, TarredFolder, GZipArchive and GZippedFile.

And when I say "easy to teach", what it really means is "find yourself a zipping example, and replace class name occurances of Zip{something} with Tar{somethong} or GZip{something}".

Sure, there are some gotchas, like the fact that a GZIP archive cannot contain filenames with subfolders, are not well-suited to contain more than one compressed file, and can contain files without filenames. But these are details you'll get used to quite easily.

There are two things that make me really proud in that product. One is under the hood, and the other is a sample. First, the engine: My colleague Jacques and I have come up with what we call the "Storage Engine". It's an abstraction of what an archiving library needs in term of temp storage, in-place archive updating, and transactional operations on an archive. Both the new TAR and GZIP implementations use it. In short, it abstracts the fact that we want to always update an archive in-place when possible, but revert to temp files and make sure to commit those temp files with any existing archive upon the last modification of it. If things go well, the ZIP implementation will benefit from it sooner than later.

Second, the sample: The FTP Sample Explorer is gone, replaced with the FileSystem Snippet Explorer, a sample that let's you see, modify and run code snippets that show you the various tasks one might wish to implement. It goes straight to the point. No bells and whistle, no gravy, just the meat. The code is embedded in the executable as compressed serialized XML data. The main information (each topic's description and code) is nothing else than rich text. The nice thing about this sample is that in order for me to modify and add new topics, I simply need to compile the project with an extra define, and I'm now running the application in "admin" mode, enabling me to update the compressed XML file directly, for the next compilation to benefit from this update.

Though I've finished work on this 3.0 version, I already have both hands in the two next releases of Xceed Zip for .NET and Xceed FTP for .NET. The first one will add support for AES encryption, and the second one will now offer proxy support.

I didn't have much time to write because all those releases have a tight schedule I can't bust. I'm leaving Xceed in two months. Yup, I've decided it was time for me to move on. Until then, I have agreed to complete AES implementation, help Jacques kick-start proxy support and train about everybody here, each earning one of the numerous hats I'm wearing. It was a very difficult decision, since I have only friends at Xceed. Though the nine years or so I've spent here were exciting and challenging, I feel it's time for me to try new stuff... by myself. This isn't a divorce. I won't be far from Xceed, and still available to help them from time to time. As for you, dear customers and readers, rest assured you will stay in good hands. The team behind Xceed Zip and Xceed FTP, both .NET and ActiveX, will remain strong, even get stronger than it is now.

.NET | FTP | General | Zip

4/5/2006 8:43:13 AM (Eastern Daylight Time, UTC-04:00)  #   
 Friday, March 10, 2006

Last Wednesday, I was invited to the Visual Studio Talk Show, a French podcast about software development. I enjoyed the experience a lot, but can only understand how you can be intimidated when it's the first time you express yourself such in a "live" way. I may have experience teaching in front of a small crowd, or presenting in front of a larger one, you cannot prepare yourself in the same way when you're an invitee, and depend on your host's questions and direction.

For example, everybody agrees that Scott Hanselman's HanselMinutes improved dramatically, no later than starting with his second podcast. That's probably why I'm left with mixed emotions. I'd repeat the experience anytime, just to have an opportunity to improve. But this time, I'd be the driver!

3/10/2006 12:07:10 PM (Eastern Standard Time, UTC-05:00)  #   
 Monday, March 06, 2006

Last week, I got a car accident. I was driving my son to the kindergarden. We were lucky, I hit the side of the other car, who was going in the opposite direction, so the impact was more friction than collision. My air bags did not open, that's a sign the impact was not a head-on collision.

My first instinct was to make sure my boy was ok. He was, top shape, asking what was the noise he heard. I moved the car in a nearby parking lot, so did the lady in the other car. Everybody seemed ok. It felt good. "It's only metal" I kept saying to myself and the lady, who was very sorry about her mistake. She did not notice the red flashing lights indicating a defective light and requiring full stop. "I thought it was green" she even said.

Anyway, my boy's fine, I'm fine, the car's at the garage and will be fixed soon (I hope), without me requiring to spend a single penny (thanks to a no-fault), courtesy car included. And to everybody I tell this story, I keep saying "It's only metal. The important thing is my son and I are ok."...

No... I'm lying... It's not only metal. It's a damn bruise on your pride. I should have made sure all was clear before turning left on that red flashing light. I should have been a 100% focused instead of being distracted by my son pointing the damaged light post to our right. I should have steered the wheel full right faster when I saw the other car about to collide. It went so fast, I'm stuck with the feeling I did nothing, I was a spectator.

Last week has been a difficult week. I've played real cool and calm with my son and wife, making sure they forget quickly. But something's broken... and it's not only metal.

3/6/2006 8:53:24 AM (Eastern Standard Time, UTC-05:00)  #   
 Monday, February 13, 2006

I've seen some strange behavior from my 3 year old son ClĂ©ment lately. I fear heredity...


2/13/2006 11:37:22 AM (Eastern Standard Time, UTC-05:00)  #   
 Thursday, February 09, 2006

Wonder what's the formula behind Xceed's build numbers? Here's the secret recipe:

( Year - 2000 ) * 1000 + ( Month * 50 ) + ( Day )

Heck, we even made ourselves an Xceed Version Yahoo! Widget!!!

Update: Until I learn how to open the ".widget" extension for downloading in dasBlog, I renamed the file to "Xceed". Just rename to "Xceed Version.widget" once downloaded.


2/9/2006 3:24:57 PM (Eastern Standard Time, UTC-05:00)  #   
 Tuesday, January 31, 2006

Ever since I've been working with the .NET framework, most of my time was spent on the System.IO namespace. I'm not a UI guy, I'm an IO guy! The most important class in that namespace is System.IO.Stream. And since it was well-designed, probably inspired by other successful stream implementations (Delphi comes to mind), it's very easy to expose features using streams.

My favorite use of streams is for pass-through streams. A pass-through stream is a class which derives from System.IO.Stream, but reads from or writes to an inner stream received at creation. It serves as a data modifyer or data analyser. When reading from a pass-through stream, it first reads from its inner stream, then processes the data read (potentially modifying it) and returns this data. When writing to a pass-through stream, it first processes the provided data (again potentially modifying it), then writes it to its inner stream.

Xceed Zip for .NET and Xceed FTP for .NET both use a pletoria of pass-through streams. The most popular is Xceed.Compression.CompressedStream, the stream responsible for compressing data before writing it to its inner stream, or decompressing data read from its inner stream. But most others are internal. We've been juggling with the idea of exposing them for a long time, but beleive it would only confuse developers to "see" those new namespaces and classes. Another useful thing with internal classes is that we can change their interface without causing breaking changes.


It was a long debate before we decided to go forth with the "transient" keyword. Not only is it used in the TransientStream type name, but also as a property on many of our pass-through streams. What we meant by "transient" is "volatile", or if you prefer more explicit keywords, "does-not-close-its-inner-stream-when-closed". A TransientStream is about the simplest expression of a pass-through stream. All required property and method overrides simply call the inner stream. The only exception is for the Close method, which simply makes sure not to call Close on the inner stream. This is very useful when you need to pass your stream to another routine which closes the stream, while you don't want your stream to get closed.


This stream does not modify the data read from or written to, but takes the opportunity to calculate either a CRC32 or an Adler32 on that data. When reading, it can also make sure, upon closing it, that the calculated checksum matches an expected stream, else throw an exception. In this way, we can insert checksum calculation anywhere in a process without interfering nor requiring code changes.


The deflate compression algorithm has the ability to detect the end of the data when decompressing. The CompressedStream is itself a pass-through stream. When reading from it, it first reads from the inner stream, then decompresses the data. When it reaches the end of the compressed data, the CompressedStream has the ability to return a stream on the remaining data, in case this inner stream contains more data after the compressed block. Why isn't this equivalent to the inner stream you might ask? Let's say the inner stream isn't seekable. The CompressedStream's Read method first reads N bytes from the inner stream, but may have found that the end of the compressed data is after M bytes (M < N). The inner stream is already N-M bytes too far. The CombinedStream receives both a byte array (the unused N-M bytes) and the inner stream as ctor parameters, and will expose those as one contiguous stream. Pretty slick!


Xceed Streaming Compression for .NET exposes stream-based (as opposed to archive-based) compression formats. Those formats all have one thing in common: they have a header and a footer. Not all of them can depend on the deflate algorithm to automatically detect the end of the stream. That's why they need to make sure to never return the first M bytes and last N bytes from the inner stream, where M is the expected header size and N the expected footer size.


When exposing part of a zip file as a single AbstractFile, we need to make sure we do not read past the boundaries of that file's data in the zip file. The WindowStream exposes a region of its inner stream as a zero-position, N-length stream.


This pass-though stream automatically encrypts or decrypts the data written or read, using the basic Zip encryption (which is as weak as me in front of a cheese cake). I will be working on AES encryption very soon, and it will most probably be implemented as a pass-through stream too!


Though pass-through streams can do much of the task, it is often better for the clarity of the code to have processing done by other classes not deriving from System.IO.Stream. The NotifyStream class exposes three events: ReadingFromStream, WritingToStream and ClosingStream. Any other class can advise for those events to intervene in the reading or writing process. This old class exists since the beginning of Xceed Zip for .NET, but it has proven very useful in the current development we are doing for Tar and GZip support within Xceed Zip for .NET.


This new class created for Xceed Zip for .NET 3.0 (Tar and GZip support) can expose a non-seekable stream as a seekable stream when reading, or at least a stream reporting a Position when writing. When reading, you can call Seek with an offset behond the current position, and it will simply read from the non-seekable inner stream until well positioned. And for both reading and writing operations, it counts the number of bytes read or written so it can report a position (granting we knew the original position when created).


Xceed FTP for .NET also uses pass-through streams. For example, the FtpAsciiDataStream wraps the NetworkStream to perform convertion of LF to CR/LF on the fly when sending a file in ASCII mode.

.NET | Compression | FileSystem | FTP | Zip

1/31/2006 9:47:29 AM (Eastern Standard Time, UTC-05:00)  #