The growing pains of NGEDIT

Hi,

After a few days, I already feel a bit more motivated and I’ve been able to put together the first alpha of ViEmu 1.2, with some features (like code folding support) that users were asking for.

I think the main reason of the lack of motivation had nothing to do with developing per se, but with the kind of life I’ve been enjoying for a few weeks during summer (lots of free time, a bit of beach time, quite a lot of interesting social life…) and the winter life, which involves a lot less of all of that. It seemed toooooo tough to get back to the working lifestyle. I guess once that is accepted, I can much better focus my energies in development.

As a side note, it turns out that it is much more motivating to work on a product once people are using it and asking for specific features, rather than working in a product before there is any user and it’s just you and your task at hand. For instance, for ViEmu, emulating all of vim seems an insurmountable task each time I read through vim’s documentation, where the requests from users and customers are usually very precise and much more manageable. And you always feel good you are giving your users the features they want.

For some reason, the end of summer, the coming back to the city, the day-job, the hangover of releasing ViEmu… I am very little productive in the last few days. I drag myself to the keyboard, but my fingers just find the way to fire up FireFox and browse the stream of stuff I usually read. Even that information seems much less interesting than other times. I drag myself back to Visual Studio, but to no avail.

I feel like I have no energy – any coding problem seems insurmountable. I know what I want to do, but I just can’t get myself to do all the little steps it takes. I can’t decide how to structure the classes, or where to store some new data. My concentration is almost non-existent.

This has happened to me other times. I guess I’ll have to let itself wear out.

1. I’ve donated 5 licenses of ViEmu to Seth Dillingham’s PMC fund, which are auctioned together with other software in order to raise money for the Jimmy Fund for cancer research.

2. Some searches such as “vi keymap Visual Studio” are already landing on the ViEmu page, as logs are showing. Not many, but they are. The Google ads I set up over a week ago are not showing. And, the main ViEmu page is already #12 in Google when you look for “vi emulation visual studio”.

3. After some input from people who are checking out ViEmu, I found out that ViEmu does not show up when the “View Code” button is pressed on the project window in VS. Given that I mostly use C++, and this button is only there for VB/C#, I didn’t even know about it. Some testers have used ViEmu with C# or VB, but it seems they didn’t use it that often. I spent countless hours to find out what was happening – mind you, the only way to track such a thing is to step through the assembly code of VS (heavily COM based code), which is quite a pain. I finally found out that the C# project managing code directly calls up the standard C# editor for such a case, bypassing the general VS editor mechanism. The only way to fix it will be to intercept all code windows in the environment, which I was already planning to do. This will also allow ViEmu to be used with multi-view editors such as HTML, but it will take some work (esp. for the OLE command stuff, which is less controllable if ViEmu is not a standalone editor but an interceptor of a standard one).

4. I’ve already checked ViEmu with Visual Studio .NET 2005. Works like a charm. A person from the VS team wanted to check ViEmu out and that prompted me to try porting it. For some weird reason, the VS2005-specific release I prepared does not install on his machine. As it works on mine, I have a hard time testing it. Sigh. I guess VS2005 being still in beta helps explain it.

5. I have some days off from my day job, so I’m out of the city, and I am using a pretty slow Internet connection. And we used to call this “Internet”? I think “lame excuse” would serve it better.

I like to get carried away with stuff. Mainly with ideas, it gives me the energy required to beat them to death until I can squeeze out the last ounce of juice. When I don’t let myself get carried away with stuff, creativity is gone.

Of course, if I’m doing something risky and dangerous, like driving or debugging, I prefer not to get carried away. It’s much better for all.

And of course, letting yourself get carried away has its drawbacks. Sometimes you get carried away with stuff that is plain dumb.

One such thing was the post yesterday. I found out that the HTML in my site was wrong and made the association with not getting search results. But it was just a thought, and it may or may not be true. I just jumped to the conclusion it was true… and posted about it.

When Ian Landsman mentioned there are probably other reasons for the fact, I revolved a bit until I actually admitted to myself that I had jumped to unjustified conclusions. Duh.

I have expertise in many areas, but web site creation and search engine optimization are not two of them.

Anyway, both the good and and the bad sides of blogs is that you get “the human factor”, mistakes and all.

I’ll post about how searches evolve… and will try not to jump to conclusions before checking them!

After a lot of work and testing, ViEmu 1.0 is finally out. You can check it at www.ngedit.com. I hope you like it. All feedback is welcome.

I think I will be able to come back to posting some interesting articles later this week.

Best!

ViEmu beta is going well. Beta 1 had some trouble, as I had left a DLL out from the package, but after several hours of fresh installs of Visual Studio I found out. Today I’ve issued Beta 2 with some improvements in Intellisense integration and better TAB handling. I also have the new web design almost ready.

I still keep the target of releasing before the end of July. We’ll see if nothing turns it into an August release 🙂

Unfortunately, this isn’t leaving too much time for technical posts, although they’ll be back after the release.

No, it’s not NGEDIT. It’s ViEmu, and it’s almost ready. Vi editor emulation for Microsoft Visual Studio .NET 2003.

Let me explain myself.

A few weeks ago, an idea struck me – I had a quite large chunk of vi emulation working. And having vi emulation would sure be a great addition for Microsoft Visual Studio (at least for those, like me, who have their fingers hardwired to vi’s input model). I had no experience writing extensions or add-in’s for VS, but it couldn’t be that hard. I gave myself a full afternoon to research it (“whaddayamean, switching to another project before the first is ready?”) and verify it was as good an idea as it seemed.

After two hours, reading a blog comment saying “if I had vi emulation in Visual Studio I’d be in heaven” and reading a note from Microsoft that yes, they had it as a feature for future versions, but no, VS .NET 2005 would not have it, I registered the domain name (actually two, viemu.com and vimemu.com, I wasn’t sure). I downloaded the VSIP SDK (add-in’s only have limited extension capabilities, you need the VSIP SDK if you really want to shake Visual Studio). And I started hacking at all that COM code.

In a couple of days, I had been able to have a custom package loaded within Visual Studio, and subclassed the editor window. I did have to learn a whole lot of COM programming (I had done some basic COM, but not the kind of COM you have to do in order to talk with a beast like VS – no, I don’t love it more now). I briefly considered C# but went with C++ instead. I did the first experiment with the following thought: “if I can make pressing 0 send the cursor to the beginning of the line, I can do the rest” (0 is the default way to send the cursor to the beginning of the line). I did it, felt great, and went to sleep (it was late).

I then started porting the vi emulation core. You know, the emulation core was written in NGS, the NGEDIT scripting language. I had to port it to C++ (no, I wasn’t going to bring over NGS scripting to Visual Studio). It took a solid four days to port all of it. No, adding the semicolons wasn’t the worst part. The emulation core was nicely separated from the editing core actions implementation, so I only had to implement a few primitives to get it working. The day I finished porting the ViEmu core, a lot of vi started working simultaneously – wonders of porting. I started using ViEmu fulltime to develop itself.

A few weeks later, a lot of vi implementation afterwards (there were many missing things yet in the NGEDIT ViEmu module), ViEmu is now almost ready. Missing things:

• The preferences section (you wouldn’t believe the COM programming required for a simple dialog with 5 or 6 checkboxes and an edit box – multiple inheritance from twelve base classes is used in MS’s sample code, only one of which is not a template.
• The installer – you must use MSI installer for it – I have to decode the MSI SDK, which I still haven’t been able to even reliably find for download – I’m waiting to see if I received my recent MSDN Universal subscription and I can better find it there.
• Solve some small interactions with Intellisense and the undo system. It’s working but there is some tricky case yet.
• General review of cursor positions after vi commands are performed. Some of them are not the same as with vim, and that should be right for 1.0.
• And beta testing. I’ve been using it myself for all development and it’s very stable, but I’m only using C++ and a simple one-byte-per-character codepage (even if VS uses UCS-2 Unicode internally). I’ll do some basic testing on Visual Basic or C#, but not the kind of testing someone who is actually developing does.

Ok, so, my expected timeframe is quite short. I definitely want to release it before the end of July. I also have to set up the web page and e-commerce system, but I don’t expect many problems with that. I expect to finish the development tasks listed above this week, and I want to run beta-testing as soon as the installer is ready (I need to deliver the beta itself as an installable package).

So, if you would like to beta test ViEmu, please drop me a line. I’m planning a quite small beta testing group. I’m mostly looking for people who use languages other than C++, codepages other than the usual US/Western Europe one, and Windows versions different from Windows 2K and Windows XP.

Someone who writes VB applications in Korean on a Windows Millenium machine would be a dream come true 🙂

I hope it will work with any left-to-right writing system (I don’t have any idea how it will perform with Arab or Hebrew bidirectional writing, and I’m not delaying 1.0 until that is ok).

Leaving out the vi/vim command line, which is not emulated, almost all of vi/vim input is emulated (including visual selection modes, etc). I’ll make available a full list of emulated features.

It is not compatible with Whole Tomato Software’s Visual Assist. I’m looking into it, but I’m not sure it will be fixed by 1.0, maybe a bit later.

Porting to VS 2005 is on the radar, of course, but 1.0 will be for VS .NET 2003. VS 2005 is only beta yet, actually.

And, rest assured, I’ll come back to NGEDIT after ViEmu is released.

While preparing the new post in the Compiler Overview series, I realized the control flow logic in the previous post was flawed. I have fixed it and updated the code in the post. My apologies. Drawbacks of writing the code without either compiling or debugging it.

I don’t want to start a flame war. I don’t want to advocate anything over to anyone. I’m just going to tell a little tale pertaining to my use of text editors.

About 6-8 months ago, I started doing most of my work on a laptop. I hadn’t used a laptop extensively, and I felt quite happy editing with UltraEdit (my editor of choice for years) or the built-in editor of Microsoft Visual Studio. Using VS to edit gives you Intellisense (a tooltip-box showing you the prototype of the function whose name you’ve just typed), a quick type-compile-correct cycle, and no time to start debugging.

You see, these laptops we have are nifty and all. Plenty of horsepower, decent screens, and you can use them on top of your lap at the sofa. There are only two quirks to me: one is that, if you add a mouse, you need a surface, which usually means a table, and a lot of the mobility is lost. The bag also starts to become bulky. The second quirk is that arrow keys, and even worse navigation keys (Home/End/PgUp/PgDn), seem to be an afterthought.

For the past ten years, I’ve never even thought that I was moving the hand across the keyboard to reach all these magic little keys hundreds if not thousands of times a day, and performing all sorts of selection/copying/pasting/navigation quickly. A common idiom: selecting the current line completely, and pasting a copy below it. I used to do that by pressing Home twice (as the first one gets you to the first non-blank character in the line and I want the whole line), then pressing Shift and holding it depressed while pressing the down arrow, then Ctrl-C, and then Ctrl-V twice (either that or Down-Arrow then Ctrl-V).

It’s more or less that. I must have made this action more times than I’ve said “hi”.

Now comes the wonderful world of laptop editing. There are many small variations of the laptop keyboard torture devised by different incarnations of Evil. Mine is an Acer with Del on the top-right corner (a good choice), Ins just left to it, the arrows in a nice little inverted T on the bottom right (not incredibly easy to find by touch, you see, I am a quite proficient touch typist as most people who spend a lot of their day typing).

The worst comes with the Home-End and PgUp/PgDn keys. PgUp and PgDn are dutifully struck there on the two little gaps of the inverted T – PgUp is above the left arrow, and PgDn is above the right arrow. This alone makes you unable to distinguish the inverted T shape by touch, which is such an aid for quickly locating it. And the mental cue of having PgUp above PgDn is lost. But the worst is yet to come: Home and End are the same two keys, as long as you press a difficult to press “Fn” key almost on the bottom left corner on the keyboard (second from the left, between the Ctrl and the Windows key).

This layout is incredibly uncomfortable. Such a simple idiom as I spoke about above turns into something like this:

– Move the hand around the bottom right corner trying to locate the keys
– Give up trying blindly and look down to locate them
– Try to locate Fn directly with the left thumb or little finger
– Give up and locate it either as the second key on the row or just look down
– Fn-PgUp (Home) twice
– Shift-Down, usually by touch from the previous position but w/o feeling reliable
– Ctrl-C (with which you lose track of where your left keys are)
– Ctrl-V twice
– Swear about these damn keyboards and try to remember what you were trying to write.

You see, a few days of this and I was bound to find a better solution.

I started thinking that there must be a way to edit text which didn’t require the Home/End/Arrow keys. I started dreaming up about text editing interfaces that would save all the hand movement. During one of those daydreams, it struck me: “remember vi, that old-school, bizarre, complex, text-editor?”. I had used vi (most probably vim) maybe a couple of times, once at the University HP-9000 Unix system and another time when installing Linux circa 1994. I promptly switched to “pico” or “joe” or one of the other more standard editors and forgot about it – not without learning that “i” helped you enter text (?!) and quitting required typing ‘:’ and then q or quit or e or exit or x or whatnot. The only way to add text to a line would be to use ‘a’ instead of ‘i’ while at the end of line, really weird But I had sometime heard that you DID NOT USE ARROW KEYS WITH VI. It resounded as a syren song in my mind. I commented it with a coworker, quite a Linux supporter, and he praised the vi input model really strong – no, he wasn’t using it most of the time having succumbed to development under Windows using plain Microsoft Visual Studio, but it was the best way of editing text known to man.

You see, I was perfectly happy with my editor(s), but suddenly the input model was a pain to use with my environment.

I looked up some resources, downloaded vim, and I started using it. First web page I printed out and kept at arm’s length for emergency:

Efficient Editing With vim – Jonathan McPherson

This is the best introduction to the vi input model I have found. First thing it does is tell you to stay away from the arrow keys and use h,j,k,l for moving around. Sure its weird, and I probably couldn’t have done it on a regular keyboard, but the laptop nightmare got me avoiding the arrow keys by sheer instinct.

It felt weird to have your hands stuck there in their home position. It felt even a bit “restrictive”, and things didn’t come out quick. It took about 2 weeks to become basically fluent with the vi model, and I was playing games of what commands to use for every little thing (“How do I erase this word? Oh, it may be, what key was it? ah, yeah, ‘b’, to go to the beginning of the word, then ‘d’ to delete and maybe ‘w’ to delete to the next word HURRAH! IT WORKED!” – then “diw – Delete Inner Word does it as well!”).

But after the steeeeeeeep learning curve, I started to become very proficient with it. You could split, resize and move around editing windows with just keyboard commands (“you mean, ‘k’ is up, but Ctrl-W and then ‘k’ gets you to the window above the current one? NEAT!”). Everything is there at your finger tips.

So – there I was, early 2005, discovering the input model of a text editor written, what, 25 years ago?, and finding out that it is the best input model for my current platform.

Mind you, once I’ve learnt it, it’s also much quicker to use on a regular keyboard – the incentive just wasn’t there with a fancy regular keyboard.

Now editing is no less comfortable at the sofa, on the plane or bus, than on a non-portable full size desk.

And do you know how I make an additional copy of the current line nowadays?

yyp

[Update May 31, 2006: if you want to learn vi/vim editing yourself, I think the best way to do it is the graphical tutorial I prepared recently. Enjoy!]

By the way, I’ve seen NGEDIT does mean something in some language (I think Indonesian, although I can’t be sure). If anyone who knows the meaning comes by, I’ll be very grateful for knowing what it means.