indistinguishable from magic (more on .net/c++ windows development)

Corey Taylor, a helpful developer on the g3d sourceforge forums pointed out some problems with code I posted last night; I think he's saying, essentially, that my C#/.NET app isn't doing what I want because the c++ code I'm calling doesn't do what I want. Makes sense. I need to make the c++ work, which means
create a valid G3D::RenderDevice, with an OpenGL context, and then access the image drawn by the RenderDevice. Arrg... There's some magical switchover between HDC (handle to a device context, I think -- a very very windows concept) and OpenGL context and System.Windows.Forms.PaintEventArgs...
Arthur C. Clarke said that any sufficiently advanced technology is indistinguishable from magic. How about any sufficiently obscure technology? Obscure, occult, obfuscated...

Aha. System.Windows.Forms.PaintEventArgs.Graphics.GetHDC() lets me access the HDC associated with my very own .NET component, in its Paint handler.

Now I need to go from an HDC to a G3D::RenderDevice, or maybe a G3D::Win32Window. I don't understand the relationship between a "window" and a "render device" in G3D. I can go read the code, though.

Meanwhile, Morgan McGuire (the uber-knowledgeable creator of G3D) is suggesting I create an ActiveX control, but I'm thinking, "Oh god, do I have to learn another Microsoft GUI component technology?"

How not to get a job over the internet

I posted in craigslist looking for someone to clean my apartment after I move all my stuff out. I liked a message from this one student; he seemed pretty clueful, and was a student. I wrote back to him with the intelligence screening (in the previous blog post) but I haven't heard back. I just noticed the subject line in his email to me:
"responce to your job add."

Sorry, no. Misspelling two words in the five-line subject of a response to a job ad is not the way to demonstrate your acumen.

Mistakes of previous professional cleaners:


  • Leaving the windows of my seventh-floor apartment wide open. I have two cats, and the windows don't have screens.

  • Sweeping the floor near the cat food, then putting the dirt from the floor into the cats' dish.

  • Using windex on a brand new (two days old!) 19" CRT monitor with an antiglare finish. Goodbye, antiglare finish.

  • Getting trapped in the bedroom by closing the door and pulling the doorknob off, then not being able to put it back in.

  • Putting my eyeglasses into a mug along with a dozen pens.

Now, is there anyone who can spell who'd like to clean my apartment for a few extra bucks?

george w. bush is stupid

George W. Bush is stupid... or at least, he has some serious problems with verbal communication. I'm watching him give a live news conference -- I don't have a transcript yet, but if I do misquote him, the misquoting still accurately represents his errors.

Re Guantanamo and Abu Ghraib prisoner abuse scandal: "We've investigated each of the detainees..." How about investigating each of the claims of abuse? And, "[the detainees are] men who've been trained to dissassemble. That means to lie." Sorry, George, "to dissemble" is to deceive, and "to disassemble" is to take something apart. You're almost right.

I love this one: "For almost a decade, the United States hasn't had an energy policy." Wait, George, weren't you president for five of those years? And wasn't your party in the majority in congress? So whose fault is it that there's been no sensible energy policy? Really hard to blame the radical left for that one. Oh, I see, the point of this speech is to wave his fist at congress because they're not completely bending over to do absolutely everything he wants.

There were more, and I'm sure we'll be seeing them on the blogs soon enough. For a comparison of intelligence requirements, I'm trying to find someone to clean my house for money, a professional move-out cleaning. I'm asking candidates to answer a quick intelligence screening, because in the past some cleaners have made some really stupid mistakes, like using windex on a brand new monitor with an antiglare coating. So here's the questions I ask:

You can tell whether it's safe to unplug something plug by...

Clean computers and electronics with...

If we're not careful, cats might...
a) run out open doors
b) fall out open windows
c) start a fire
d) exert mind control on their owners

Business cards and receipts piled on the floor in a corner are probably...
a) garbage
b) recycling
c) precious
d) no idea, so i'd better save them

Pick the right word... "Gee, this diet must be working, because my
pants are really (a) loose. (b) lose."

I ask you -- could George W. Bush have answered each of those questions correctly? If he can, please send him to Providence; I need to hire some unskilled labor.

sample p/invoke VS.NET 2003 solution

Microsoft includes a sample of language-interop via p/invoke with Visual Studio .NET 2003, but it's only set up as a makefile project. I do most of my Windows coding from inside VS.NET, so I wanted to learn how to create a dll from C++ code, then call it from C#, all from within the IDE. I built a VS.NET 2003 solution which builds the Microsoft sample code from within the IDE. Just posting it in the hope someone else might find it useful.

What I learned is that there's a very particular set of compiler options which must be set, and some very specific keywords/tags/attributes in both the C++ and the C-sharp code.

To set up the dll to have exported functionality:


  1. Set the compiler options so they look like this: /Od /D "WIN32" /D "_DEBUG" /D "_WINDOWS" /D "_USRDLL" /D "PINVOKELIB_EXPORTS" /D "_WINDLL" /D "_MBCS" /FD /EHsc /RTC1 /MTd /Yc"stdafx.h" /Fp".\Debug/PinvokeLib.pch" /Fo".\Debug/" /Fd".\Debug/" /W3 /nologo /c /Gz /TP. To do this, edit the Property Pages for the project. Go through each of the C/C++ subcategories and set the options to get exactly this command line listed in the "Command Line" subcategory. skoolLIB.vcproj has these settings already, if you want to copy from there. This seems to do a whole bunch of things, including generating precompiled headers and a dll.

  2. Tag the declaration functions you wish to export with a macro -- this is too insane to describe; see the first comment in skoolLIB/PinvokeLib.h for details.

  3. Export the data structures: 
    extern "C" PINVOKELIB_API int TestArrayOfStructs( MYPOINT* pPointArray, int size );
    typedef struct _MYPERSON
    {
     char* first; 
     char* last;

To use the exported API from C#:


  1. Use the StructLayout attribute to describe the layout of datatypes to import: 
    [ StructLayout( LayoutKind.Sequential, CharSet=CharSet.Ansi )]
    public struct MyPerson 
    {
     public String first; 
     public String last;
    }


  2. Declare the functions to import, and indicate where to find the library from which to load them: 

    public class LibWrap
    {
    ....
     [ DllImport( "..\\skoolLIB\\Debug\\skoolLIB.dll") ]
     public static extern int TestStructInStruct( ref MyPerson2 person2 );
    ....
    }

  3. Call the imported functions as if they were normal functions:

     int res = LibWrap.TestStructInStruct( ref personAll );

Compiling the c# files doesn't need any special options, because the complicated part -- finding and loading the DLL -- is done at runtime. You've compiled a relative path into your code, though, which will probably complicate distribution.

I'm interested if anyone has insight into this process, or can point me at better documentation than what I've just written. There's way too much documentation; I guess I've just compounded the problem.

oh, the joy won't stop.

I was getting somewhere with the plan I laid out in my last post... except I couldn't get my C# code to call a function exported in the sample unmanaged C++ DLL created by VS.NET; I'd get a System.DllNotFoundException when I tried to call the function. I tried putting the dll into various places where Windows is supposed to search for dll's, but not dice. I decide to restart the computer -- a nice soft reset, quit everything nicely.

When the system comes back up, I launch the VS.NET solution file with all the various projects I've been working on... and VS.NET hangs. Okay, I launch it again... and VS.NET hangs again. I launch it again... and then give up. I didn't need that solution anyways.

I decide to find a complete working example of just the part that's troubling me, calling unmanaged functions from managed code... The example I find is actually already on my computer, in C:\Program Files\Microsoft Visual Studio .NET 2003\SDK\v1.1\Samples\Technologies\Interop\PlatformInvoke\Custom\. Why didn't I look there before? This sample is set up to build directly from the command line, with makefiles. Makefiles! Lovely! With a makefile, I can directly change command line options, and ellide all but the vital ones. Amazingly, all it takes to make a dll is cl.exe /clr /LD MyClass.cpp... Poof, there's MyClass.dll. Amazing. This one little example seems to have everything I need. Too bad it's almost midnight. Every day brings its own surprises.

progress with windows and opengl... towards a solution!

SharpGL works, and I'm learning about how to call unmanaged C++ from managed C#. It's all about loading libraries and those windows-specific tags. First there's a class, Imports, whose job is specifying and importing all the dll's, and establishing the structure of a few important types, such as PixelFormat. With that accomplished, a class called SharpGL.SceneGraph.OpenGL imports most of the OpenGL functions, and then wraps them with a public interface.

This line, from SharpGL... 


public class OpenGL : Imports {
....
[DllImport(LIBRARY_OPENGL)] protected static extern void glVertex3f (float x, float y, float z);
...
}

...means that I can do this, in C#:
 
OpenGL.glVertex3f(x, y, z);

...but that's a protected function, so what I really need to do to call this code from another package is

myOpenGL.Vertex(x, y, z);

This is good, this is progress, but I don't want to write a molecule renderer in straight OpenGL. SharpGL provides a .NET component with a scene graph and lighting controls and camera controls and built-in shape classes, which is very helpful... but I want to use G3D, not SharpGL. G3D exists, and I know it, and I know its maintainers, and I know it is fast, and I know it works. Corey Taylor (one of the maintainers) says he'll try to help, but he's under a crazy workload... so I'm going to keep trying to do this myself.

My current approach:



  1. Keep my current C# class, MolecularModel, as the data structure for describing molecules. Unmanaged C++ can access managed, public C# classes.


  2. Write a small, unmanaged C++ class which renders a MolecularModel to an offscreen bitmap using G3D. Tag just the public interface of this class as dllexports.


  3. Write a small managed C# class which wraps my unmanaged C++ molecule renderer, by dll-importing the public interface which I exported in step 2. This managed class will not be a control; it's just a wrapper class. This wrapper's fundamental functionality will be to render a MolecularModel into a Bitmap.

  4. Replace the internals of the current ChemPad molecule viewer control, which is a .NET component, with calls to the managed molecule renderer wrapper. On Paint events, draw the Bitmap created by G3D onto this control's Graphics. Now instead of jmol-net generating the 3D image, G3D will be generating the 3D image.

This approach is different from my previous attempts in that it adds a layer which exposes a small API from unmanaged C++ to managed code. The compiler options will still be fairly hellish, and this isn't as good as really making all of G3D accessible from C#... but it keeps a very clean boundary between the chemistry part of the application and the rendering part of the application. Yikes. Wish me luck.

Windows Programming Sucks

Now that I've made my decision about where to work, I can finally come out and say it: Windows Programming Sucks! It's okay when everything is packaged up nicely, or when there are step-by-step instructions or examples for exactly what I need to do... but when I'm trying to do something just a bit odd, it's hopeless. Listen, I can RTFM with the best of them, but with these giant Microsoft products, RTFM ends up referring to dozens of msdn articles, book chapters, forum posts, and worst of all, tiny "help" strings (as if) in a multitude of property inspectors. Those articles etc depend on a deep knowledge of how Windows programming worked in the last two or three iterations: COM vs COM+ vs ATL vs MFC vs OMFG!. I swear to god, the solution to my problem seems to be "Just use P/Invoke." Then it turns out that P/Invoke is actually pinvoke and it's something that gets called by IL (Intermediate Language) which is generated by the C++ compiler. So "using P/Invoke" actually means something like "In these six dialog boxes, set these nine options to these values: '/Gz', 'Native', 'Never', 'Yes', and 'Only on Tuesdays.'" Except the documentation doesn't say so directly.

So why on earth am I using Windows, anyways? I'm working on a project which makes heavy use of the Tablet PC digitizer and Microsoft Handwriting Recognition. No other platform delivers these capabilities. The Tablet PC digitizer, when used with the Microsoft.Ink api's, generates an order of magnitude more samples of stylus location than external Wacom tablets, or any other device I've used. This level of detail yields much better handwriting recognition results. So, yes, thank you for that, Microsoft Tablet PC team.

My task for the last few days has been trying to switch renderers for this application, from a pure-C#, non-hardware-accelerated, non-standard chemistry renderer, to an OpenGL renderer, specifically, G3D. G3D is a multi-platform OpenGL abstraction written in C++. It's sweet, fast, well-documented, and I've been using it for about a year for various projects on Windows and Linux. So I want to use the G3D OpenGL renderer in my existing C# .NET application. In other words, I want to use a legacy library to enhance my .NET application; surely this is a common situation. Turns out that it's such a common situation that there are at least four ways to do it, and none of them are explained clearly, anywhere.

The best two options seem to be making a managed C++ wrapper which calls the "unmanaged" G3D code, or recompiling G3D with the "/clr" flag, which means, compile it to intermediate language, to run on the common language runtime. Sounds good, but the documentation for the first option is all about calling code in an existing DLL, or dynamically loaded library. G3D only exists as a statically loaded library. In Windows, a variety of insane keywords (__cdeclspec dllimport!) are required to tag any parts of an API which should be available to users of a DLL; those keywords are not required for users of a statically-linked library. G3D doesn't have those keywords, and so G3D doesn't make a useful dll. I haven't found documentation on calling statically-linked unmanaged C++ code from managed C++.

Recompililng G3D with the /clr flag sounds good, except that it just pushes the problem off by one step; G3D statically links with other unmanaged C++ libraries, like libpng and libjpeg. So I'd actually have to compile all of these satellite libraries with /clr, too. I haven't tried this yet, but I have a feeling that those libraries will try to do things which aren't permitted in managed C++; they're tuned for fast image manipulation. Even leaving out that worry, adding /clr to the compiler options for the G3D code triggers a cascade of incompatible compiler flags: run-time checking, exceptions, and certain kinds of optimizations, don't work with the /clr option. To turn off those incompatible compiler flags, though, requires the use of, I kid you not, three levels of hierarchical UI: per-project, per-category, and per-subcategory, with a dozen options for each subcategory. The subcategory options have names that don't necessarily correspond to the flag they modify; ie, to get the /clr flag, I set "Use managed C++ extensions" to "Yes," and to turn off /Gz I actually have to set two separate run-time type checking properties to some unknown pair of values. In fact, this is where I'm currently hung up; I have to turn off /Gz to compile with /clr, but I can't figure out how to turn off /Gz. The options in the three-deep UI for setting one of the two run-time type checking flags are something like "Auto," "Default," and "Yes," but what I need is "No."

I'm disheartened. None of the approaches I try quite exactly fail, they just descend into a combinatorial explosion of options and an ever-widening mass of documentation.

My remaining options:


  • using an existing C# OpenGL library: Tao, which is part of Mono, which is a whole 'nother barrel of fish; it seems to include its own compiler, and so I have no idea whether the Microsoft.Ink stuff will work with it. I dobut it.

  • using another existing C# OpenGL library: SharpGL. This seems like it might work, but the license looks very casual, and I got into this whole mess because I wasn't paranoid enough about the Java Research License.

  • Switching my application to be mostly unmanaged C++, with the Ink stuff in a managed C++ component. This is probably the most straightforward architecturally, but a) it requires rewriting a fair amount of code just before I turn maintenance over to a colleague and leave this job, b) it seems like an admission of defeat, and backwards, and c) it just might send me into the aforementioned dll hell.

I have to complete this task before I can leave this job and move to California. My frustration with Windows programming is unbounded.

whiteboards of the CIT during finals

I'm on the third floor of the computer center (aka the CIT) where the computer science department has just completed amazing renovations. The third floor used to be all offices and belong to the IT people. Now the CS department has taken over, and built such a fantastic space for studying and hanging out and working that students from all departments seem to have found it and made it their study nook. On the whiteboard covering the wall to my left, a large diagram describes the structure of a "Weird Detective Story," with sections like, "Other Characters: Woman, a god, Theodore Roosevelt's ghost, a corpse," and "Context: Several dimensions - in the physical, not time-travel sense. Resurrection of old social movements? Underground and cabalistic." The whiteboard in front of me has a careful top-down view of a string of railroad cars negotiating an s-curve, paired with what appears to be the face of a seal: round and whiskered, but without ears. Do seals have ears? Off to my right is a diagram of the evolution of animals; most of it is too small to read, but the diagram is capped with a manga-ish drawing of a creature with a big belly, labeled "Placental Mammal: Long Live Toro the Forest Spirit!!!"

Brown is... special. I'd like to know which book the lit-crit kids were analyzing... and I bet there's a story behind Toro the Forest Spirit. Mostly I have to congratulate the CS department on creating such a beautiful, usable space. For my last weeks at Brown, I think I'll make this my home.

thoughts on laszlo

Ah. I spent much of today talking to some very interesting people at Laszlo:
David Tempkin is the CTO and a co-founder; Sarah Allen is a co-founder and a hardcore programmer; and Scott Evans is a developer. I also met with Jamie Clark, Andrew McClain, and Pablo Kang, but they don't seem to have weblogs. These are really good people. Good people.

I did a little exploration of the Laszlo platform... i made a little rss presentation widget. There's not much to it; it's maybe ten lines of xml, with no error checking... but the technologies just fit together: rss and lzx, the lazlo language.

I'm very, very happy. I've promised myself, and someone else, that I won't make any commitments until Tuesday. The next few weeks are going to be challenging and uncertain, and in some ways I'd like to end the whole process right now by saying "yes! sign me up! put me to work at laszlo!" but I know that would be a short-circuit. Just to make the right decision isn't enough; I need to make the decision rightly.

top three technical skills

Recruiters keep asking me my top three technical skills. Let's see if I can figure it out -- here's an unordered list.

  • Object-oriented design and programming
  • Specification-oriented interface design and implementation
  • Disciplined software engineering: checking for and reporting error conditions with useful error messages, meticulous use of version control, commitment to clean design
  • Taking advantage of language and library features
  • Tracking bugs and odd behavior to their source by reading code
  • Finding, evaluating, learning, and using new tools: libraries, applications, utilities, books, tutorials, etc.

  • I don't think these will make much of an impression on recruiters, but I think a programmer would appreciate this list. Any suggestions?

national holidays: a proposal

There are some routines which I think should be added to our national yearly calendar of things to do. April 15 is tax day; there's Mother's Day and Secretary's Day and people in LTR's tend to celebrate anniversaries... but what about Clean All Those Unused Icons Off Your Desktop day? I think the current calendar leaves out some rather important and often neglected activities of the digital lifestyle.

My proposals:


  • Change Your Passwords Day Sure, it'll be harder on the infrastructure, not to mention a security risk, if everyone changes their passwords on the same day -- but isn't that better than everyone using their kid's birthday for all their passwords ever?

  • Shareware Appreciation Day On this day, we should all demonstrate appreciation for the authors of open-source software/shareware/freeware/peepsware/etc. We'll demonstrate this appreciation either by sending props, peeps, or payment to the authors. Come on; how long have you been using that copy of Stuffit/SecureCRT/Transmit without registering it? Don't the authors deserve at least a shout out?

  • Backup Day I backup all the time, okay? If I'm working on an important document, I've almost always got it under version control, with the CVS server running on a professionally-maintained, daily-backup'd file server. (Thank you, Brown Department of Computer Science.) But most people just... don't. Eventually their computer crashes (if it's windows, which it usually is) or they switch jobs, and their old data just disappears. Regular backups are the answer but mortals don't do them until they've been through the pain of a hard crash. Backup Day will encourage and support all computer users in backing up their data. It'd be good for everybody. Maybe Symantec and Retrospect could get together and promote Backup Day.

  • Clean Your Hard Drive Day Search out those unused applications, that junk mail file, coredump files, used podcasts, bittorrent downloads that never finished... and delete them! Then defrag your hard drive.

avalon eyecandy

There's some eye candy to avalon. In fact, I've seen a few things that look pretty fantastic. Smooth gradient shadows, two-and-a-half-d layering effects, liberal use of rounded edges, large san-serif type... It almost looks like... Mac OS X!
The applications that I saw at conviat.com are a particular theme for Avalon, fine, but it makes me happy that Avalon is capable of that sort of visual feel. I use Stardock Object Desktop to skin my tablet pc, which makes the look acceptable, but there's a hell of a performance cost. Now, if Longhorn actually builds in skinnability at the very lowest level, and I can actually skin my environment with no noticeable performance overhead, well, that might be a system I could work with.
As it is, yes, I'll say it right here and out loud: I choose to use my 15" Mac PowerBook G4 867 MHz over my 1.5 GHz/1 gb ram tablet PC and my 2.2 GHz giant dell laptop with a 1600x1200 screen because... it's more pleasant. It's just more pleasant to use the mac. It doesn't have to always be that way. If I joined the Avalon team, I could help make Avalon more pleasant, and I could help improve the user experience of the largest segment of personal computer users in the world. I think that could get me out of bed in the morning and staying late at night.

dashblog / dashboard / Tiger

I am writing this here post from my dashboard, via dashblog. I get the feeling that dashboard might replace Quicksilver; it's kind of easier to get at things when they have a user interface which reveals their abilities. Could I post to my blog via quicksilver? Well, probably, but I don't know how -- whereas, I just clicked on a link to download and install this dashblog widget, and now here I am. My next task is to find the perfect del.icio.us post/search widget.

a decadent morning

I climbed into bed just after Jon Stewart said, "Okay, people, that's our show for today! Goodnight! Here's your momoent of Zen!" Then he showed a clip of Bruce Willis, bald but with a Bolton-style mustache, reading a speech in exceptionally bad French. (I don't speak French, but I could still tell it was bad French.)

I woke up at 7:10, or actually, 7:03, because the clock in my bedroom is seven minutes fast. The alarm was set for 8:30, so I knew I had some time for a mellow morning. I drank coffee with cream, spread nutella on wheat bread, and read Harper's Monthly. Ah. Just delightful.