The growing pains of NGEDIT

I’ve just released ViEmu 1.2 a while ago. During the last weeks, I feel fully energized, and I’ve completed quite a lot of work.

I haven’t seen almost any traffic at all coming from the new articles, but then, Google doesn’t like my page a lot, so it’s understandable. Maybe I am in the dreaded “sandbox”, or maybe I’m missing some key stuff in the page. My consolation is that anyone who looks for “vi keystrokes visual studio” or anything remotely similar will decidedly find it through the many mentions that appear in the first search results page (my page is nowhere to be seen on the first 20 or so pages).

I thought I’d share my plans for the next steps, especially since I’m not working a lot in NGEDIT in the last times.

Well, actually a lot of the current work will reflect directly on NGEDIT. I already have the regular expression framework that will power ViEmu, but the good thing is that it’s written using the C++ string classes I talked about a while ago, so it will transplant directly to NGEDIT’s multi-format text processing engine (even if ViEmu only uses UCS-2 two-bytes-per-char support). I will be porting the latest code I did for NGEDIT, which is among the most innovative stuff, to this string support, which is evolving within ViEmu.

My intention is to focus on ViEmu for a while more – until I get it to a level with which I will feel comfortable. That means, basically, customers’ requests and vi (ex) command line emulation. NGEDIT is a product with much more potential, but I feel I need to give ViEmu enough gas so that it will be able to work well. It’s very motivating to work directly on customer’s requests. And the fact that iit is already able to generate income is also a great incentive (when comparing it with NGEDIT, which will stil take some time until it can become a product).

I will estimate a time frame, given that I can always explain later while I missed badly 🙂 I calculate that a bit over one month will be enough to get ViEmu to my desired level, and then I will be able to invest much more effort on NGEDIT while still improving ViEmu.

I’ve been very busy during the last week. For one, I’ve been working hard on ViEmu 1.2. It is now, hopefully, in the last beta stages, and should be out in a few days. I’ve corrected many things in previous versions, and it now provides much more reliable vi-style repeatable input, macros, etc… even if the focus was to get code folding and basic window navigation support, in the end it has become a release that “stabilizes” many features.

But during the last few days, I’ve finally made a general upgrade of the web site. Following the suggestion from reader Lena (thanks!), I found out about Cheetah, and I’ve been able to finally prepare a web site development framework that makes sense to me. Read on if you’d like to know more details.

What have I done to the main site? First, I have added a much needed ViEmu FAQ section. It answers the most common questions I get about ViEmu.

Second, I have changed the “tone” of some of the pages (about, etc…) getting rid of the ugly “About us”, “We are…”, etc… tone. I think there is no point in keeping a “corporate” look, especially since I am quite clear in the blog and I don’t really make an effort to keep a corporate face – having the web site like that was the simplest thing to do when I first prepared the web site.

And third, the most interesting – I have added an Articles section to the web site. There are links to the older posts in this blog which can be interesting, but I have also written about ten new short new articles. You can find one describing how I’ve used Cheetah to structure the web site (be sure to check it, if you have some similar problem!), and some other articles on Visual Studio extensibility, learning vi/vim, etc…

On one hand, there are a lot of useful articles I can write quite easily, which wouldn’t make sense as blog posts. They are fine there for anyone searching for the topic. On the other hand, I think they will help drive some traffic to the main NGEDIT site, and I hope that will increment the exposure of ViEmu (and, eventually, NGEDIT).

I will be posting references to new articles on the blog, apart of course from all the regular postings sharing my evolutions of the products and the company.

I have finished implementing and packaging ViEmu 1.2, and sent out an initial release to current customers and interested users. It includes folding support and window-command support as in vim (I think none of these was in the original vi). By the way, it is already using the C++ string class I talked about in the last post – not heavy use yet, but already using it. After a bit of testing of this release candidate (as the version has already been released), I will be announcing it and putting it up for download on the main page.

The main web site is built with simple, static html files. There is quite a lot of repetition, both for common elements like the navigation bar and for common parts such as general layout. I guess that must be the case with many sites. I’ve been wanting to add two new sections to the web site during the last weeks, but having to update those elements on all pages was not something I wanted to do. I am going to set up a sensible framework such that those elements don’t have to be updated in many places.

I think many sites use a dynamic mechanism, such as ASP or PHP, to avoid replicating such elements. This blog, for example, which is based in WordPress, does such things. But I do not want to switch the whole site to a dynamic system – it seems absurd to evaluate code in each page-request when it can be a one-time-off that generates the proper html files.

I do the html and css by hand, using vim, and I like to have that kind of control. I don’t know of any system that provides what I want – some kind of “macro preprocessor” for html pages. My idea is that I will be writing “.hs” (“html source”) files, and a preprocessor will be preprocessing them to generate the actual html files. There will be a “.hi” (“html include”) file with the common element definitions.

It’s not that I like to do stuff from scratch, but I’ve never heard of tools to do such a thing. I’ve checked the “m4” macro preprocessor, but the main problem I see is that it is leant towards single-line macros – and most definitions that I’d be using will be multi-line. It need be comfortable to use in such a case.

Unless I find out about a tool that does this, I will be writing it myself. It should only take a couple of hours to get it working. If you know of such a tool, I’d be very grateful if you leave a comment here.

It’s good to see how, as months pass, I’m getting to automate common tasks and the general “work environment” is better every week. Starting from scratch, you have to live with many cumbersome methodologies for some time, but if you are patient it’s very satisfying to improve each part little by little: I can already develop text-encoding-independent text-processing code, I will be able to restructure the web site easily, … I’m dying to develop a dual installer for Visual Studio 2003 / Visual Studio 2005 (for ViEmu) and take out another thorn!

No, the title is actually not mine – although I loved it when I read it. Read on.

One stumbling block I’m finding is that desperately want to write code which is common to both ViEmu and NGEDIT. I mean, code that deals with text. And this means all areas in which I’m developing, vi emulation, the new features of NGEDIT, etc…

On one hand, ViEmu works with the internal text buffer of Visual Studio, which work with 16-bit wchar_t, called simply ‘Unicode’ by Microsoft, but which are actually the UCS-2 little-endian encoded version of Unicode (I talk about 16-bit wchar_t’s because this is actually not defined in the C++ standard, as happens with all built-in types, and GNU gcc for example implements them as 4-byte values).

On the other hand, NGEDIT deals with all types of underlying text encodings, including Unicode, platform-native one-byte-per-char, variable length UTF-8, etc… The underlying text store management code stores the text in fixed-size pages and is accessible through simple byte pointers and unsigned count of bytes – not null-terminated strings.

I’ve been working in some template classes to manipulate strings for months now. They are unfinished and only partly usable. The deal is, when you start actually trying to build complex template stuff in C++, it gets hairy very soon – and you find out about the many limitations of C++’s templates. Not only the syntax, but the sheer amount of red-tape you have to write.

I think I’ve made some significant advance in this area. My goal is to be able to write, for example, a tokenizer, in such a way that I can instance it as TTokenizer and use it right away, the same as for TTokenizer, etc… I’ve been after this for quite some time and I’ve ended up with my head spinning several times. One other reason for the problems is that I don’t want to duplicate too much code, and TCountedNativeCharString and the other ones are also templates on more basic types.

Anyway, I was Goo^H^H^Hresearching earlier this week, and stumbled into a similar initiative by Ryan Myers from Microsoft. He’s been documenting it in his blog, and although it is unfinished yet, the 8 posts he did early this year are a very interesting read for C++ programmers. His goal is not exactly the same as mine, as he is developing a single template-based string class that manages both interpretation and memory needs, and I have separate assorted template classes for each nead – I think it wil be better for my code to separate those two. But his blog posts were an awesome read. His own words: “I’m out to create a string class for C++ that doesn’t suck”. And I couldn’t agree more.

I’m hoping that I will have the template stuff working by the end of the week, and this will in turn unlock my working in the common regular expression for both applications. I will also be rewriting the new features I’ve started implementing for NGEDIT using the new common framework (among other things, like the fact I hate writing code bound to work with ‘char’s or ‘wchar_t’s, the features may some time make it into another VS .NET add-in).

Hi,

After a few days, I already feel a bit more motivated and I’ve been able to put together the first alpha of ViEmu 1.2, with some features (like code folding support) that users were asking for.

I think the main reason of the lack of motivation had nothing to do with developing per se, but with the kind of life I’ve been enjoying for a few weeks during summer (lots of free time, a bit of beach time, quite a lot of interesting social life…) and the winter life, which involves a lot less of all of that. It seemed toooooo tough to get back to the working lifestyle. I guess once that is accepted, I can much better focus my energies in development.

As a side note, it turns out that it is much more motivating to work on a product once people are using it and asking for specific features, rather than working in a product before there is any user and it’s just you and your task at hand. For instance, for ViEmu, emulating all of vim seems an insurmountable task each time I read through vim’s documentation, where the requests from users and customers are usually very precise and much more manageable. And you always feel good you are giving your users the features they want.

For some reason, the end of summer, the coming back to the city, the day-job, the hangover of releasing ViEmu… I am very little productive in the last few days. I drag myself to the keyboard, but my fingers just find the way to fire up FireFox and browse the stream of stuff I usually read. Even that information seems much less interesting than other times. I drag myself back to Visual Studio, but to no avail.

I feel like I have no energy – any coding problem seems insurmountable. I know what I want to do, but I just can’t get myself to do all the little steps it takes. I can’t decide how to structure the classes, or where to store some new data. My concentration is almost non-existent.

This has happened to me other times. I guess I’ll have to let itself wear out.