Creating BNX clone for IRC

[nslay]

I'm going to attempt to recreate as much of BNX as possible for IRC.

Let me know if you're interested. I've created an SVN repository for it and committed some reference materials (namely BNX 1.03 files and Greetbot files).

I think I'll first create a rudimentary battle.net CHAT emulator (designed for one-on-one CHAT). This way we can use Wine to try BNX out. Though BNX is actually quite well documented.

I think the best choice of implementation language is C++ and we can use libevent for portable event-driven timers and socket multiplexing.

Again, if you're interested, just PM me a SVN username/password and get a dynamic domain name for your development machine (so I can add you to the firewall).

RFC 1459

[nslay]

I've decided to make a base IrcClient class to take care of all the basic IRC details.
So far, I've implemented all the socket code, the connect/reconnect code, basic event dispatching, and line buffering. Everything is non-blocking and revolves around libevent (this allows several instances of IrcClient to coexist).

Next is to implement more of the get/set functions (only basic ones exist), IRC protocol parsing and high level IRC event dispatching.

One of the most irritating aspects of IRC is the begin, content, and end numerics. It makes the programming a little trickier since you have to keep track of the object of interest currently being constructed. For example, the channel list is sent in multiple messages. You can be in several channels at a time as well as querying channel lists of which you are not in.

In the distant future, I'll create a derived BnxBot class that just implements the BNX behavior.

[nslay]

I wrote the IRC protocol parsing today ... I tried to stick to the RFC as close as possible. However, I'm not very familiar with the IRC protocol

I'm actually surprised it could be written so concisely! Here's what I wrote:

Code: [Select]

char * PopToken(char *&str) {
        str += strspn(str, " ");

        char *tmp = str;

        str += strcspn(str, " ");
        if (*str != '\0') {
                *str++ = '\0';
                str += strspn(str, " ");
        }

        return tmp;
}

void IrcClient::OnProcessLine(char *line) {
        const char *pPrefix = NULL, *pCommand = NULL, *pParams[16] = { NULL };
        unsigned int numParams = 0;

        puts(line);

        if (*line == ':')
                pPrefix = PopToken(line)+1;

        pCommand = PopToken(line);

        while (*line != '\0' && numParams < 16) {
                if (*line == ':') {
                        pParams[numParams++] = line+1;
                        break;
                }   

                pParams[numParams++] = PopToken(line);
        }   

        if (isdigit(pCommand[0])) {
                // This is probably a numeric

                int numeric = strtol(pCommand,NULL,10);

                OnNumeric(numeric, pParams, numParams);

                return;
        }   

        if (!strcmp(pCommand,"PING")) {
                OnPing(pPrefix, pParams[0]);
        }   

}

Does it look OK? Do you suppose there is any case that I don't handle adequately?

[nslay]

Well, I setup all the callbacks for all word commands and I setup automatic nickname handling (collision and in-use sorts of issues).

I also found RFC 2812 and added more #defines for newer RPL and ERR numerics.

I'm thinking about how to implement the BNX wildcards. Here's a snippit from BNX.TXT

? * - matches 0 or more characters of any type
? ? - matches exactly one character of any type
? % - matches any number of non-space characters
? ~ - matches at least one space character
? \ - the literal escape character

UNIX-like systems usually provide fnmatch(3) which only covers * and ?.

This sort of wildcard matching will also need to be implemented for the access system anyway (for hostmasks).

I'm also thinking about easy and portable configuration file formats. INI seems to be very simple and sufficiently flexible for this application. There is even a library called iniparser.

Any comments?

[rabbit]

Well, I setup all the callbacks for all word commands and I setup automatic nickname handling (collision and in-use sorts of issues).

I also found RFC 2812 and added more #defines for newer RPL and ERR numerics.

I'm thinking about how to implement the BNX wildcards. Here's a snippit from BNX.TXT

? * - matches 0 or more characters of any type
? ? - matches exactly one character of any type
? % - matches any number of non-space characters
? ~ - matches at least one space character
? \ - the literal escape character

UNIX-like systems usually provide fnmatch(3) which only covers * and ?.

This sort of wildcard matching will also need to be implemented for the access system anyway (for hostmasks).

I'm also thinking about easy and portable configuration file formats. INI seems to be very simple and sufficiently flexible for this application. There is even a library called iniparser.

Any comments?

Convert them to regular expressions.

[nslay]

With regex in C++11, I guess it wouldn't be an issue of portability.

The usual DOS wildcards still need to be supported for hostmasks (nobody writes hostmasks in regex).

[nslay]

It's been a while. I've been very busy these past two weeks.
Anyway, I spent some time today to implement the IRC wildcards.

Here's what I wrote:

Code: [Select]

bool IrcMatch(const char *pPattern, const char *pString) {
        if (pPattern == NULL || pString == NULL)
                return false;

        const char *pPatternLast = NULL, *pStringLast = NULL;

        do {
                if (*pPattern == '*') {
                        pPattern += strspn(pPattern,"*");
                        pPatternLast = pPattern;
                        pStringLast = pString;
                }   
     

                if (*pPattern == '?') {
                        if (*pString == '\0') {
                                if (pPatternLast == NULL || *pStringLast == '\0')
                                        return false;

                                pPattern = pPatternLast;
                                pString = ++pStringLast;
                                continue;
                        }   
                }   
                else {
                        if (*pPattern == '\\')
                                ++pPattern;

                        if (*pPattern != *pString) {
                                if (pPatternLast == NULL || *pStringLast == '\0')
                                        return false;

                                pPattern = pPatternLast;
                                pString = ++pStringLast;
                                continue;
                        }   
                }   

                if (*pPattern != '\0')
                        ++pPattern;

                if (*pString != '\0')
                        ++pString;

        } while (*pString != '\0');

        pPattern += strspn(pPattern,"*");

        return *pPattern == '\0' && *pString == '\0';
}

So as an explanation:

• The beginning is assumed anchored. So consume as many characters as you can, up until '*' or '\0'.
• Once you do encounter a '*', the algorithm reduces to something like strstr(). You just remember the last matching pattern and string position.
• A mismatch merely resets the current string and pattern pointers to the last string position + 1 and the last pattern position.
• Encountering another '*' merely updates the last matching pattern and string position.

This is pretty straightforward. However, when you add the BNX wildcards % and ~, then you have to do something more sophisticated. You need to keep something like a stack of pattern partitions to match. Unlike IRC wildcards, you may have to revisit an earlier partition since the % and ~ require some specific structure.

[nslay]

I wrote the response engine today. It's quite elegant. Have a look:

Code: [Select]

class BnxResponseEngine {
public:
        BnxResponseEngine() {
                AddDefaultResponse("Please go on.");
                AddDefaultResponse("You must be joking!");
                AddDefaultResponse("Are you talkin' to me?");
                AddDefaultResponse("Get outta town!");
        }   

        bool LoadFromStream(std::istream &is);
        void SaveToStream(std::ostream &os) const;

        void AddDefaultResponse(const std::string &strMessage) {
                m_vDefaultResponses.push_back(strMessage);
        }   

        const std::string & ComputeResponse(const std::string &strMessage) const {
                std::vector<BnxResponseRule>::const_iterator itr;

                itr = std::find(m_vRules.begin(), m_vRules.end(), strMessage);

                if (itr == m_vRules.end())
                        return m_vDefaultResponses[rand() % m_vDefaultResponses.size()];

                return itr->ComputeResponse();
        }   

        void Reset() {
                m_vRules.clear();
        }   

private:
        std::vector<BnxResponseRule> m_vRules;
        std::vector<std::string> m_vDefaultResponses;
};

By the way, the BNX executable contains no obvious trace of those default responses (at least from strings(1) point of view). The author must have done something clever to prevent hex editing.

Here are some screenshots of some classic responses:

As you can see, "/me" is not properly handled. Actually, there is no "/me" command on IRC. Only the kludge called CTCP. So, I'll have to add some battle.net command to CTCP conversion ...

EDIT: Use thumbnails instead.

[nslay]

I wrote a CtcpEncoder, CtcpDecoder and added some limited support to BnxBot. It now responds to the CTCP VERSION command (though, it doesn't have any real versioning yet).

Today, I created a sourceforge project for it:
http://sourceforge.net/projects/ircbnx/

While sourceforge doesn't think there are any files, you can check them out with subversion (I recommend TortoiseSVN on Windows).

For Windows:
EDIT: Redundant
Some porting is still necessary despite using libevent2. For one, FreeBSD's libstdc++ doesn't support C++11 std::regex and so I used POSIX regex. BnxResponseRule is the only place where regex is used and it would probably be trivial to port this to C++11 std::regex (e.g. I think VS 2010 and 2012 have this).

• regex
  FreeBSD's libstdc++ doesn't have C++11 std::regex and so I resorted to POSIX regex. BnxResponseRule is the only place where regex is used and it would be trivial to port it (it's just a header)
• Winsock
  This project is not yet aware of Winsock and it uses some POSIX functionality (like fcntl) not present on Windows. It even uses signal() to ignore SIGPIPE (which I'm not sure if Windows supports). You'd have to set the socket as non-blocking some other way in Windows.

A list of things that remain to be done:

• RPL_ISUPPORT
  At present, I assume ISO646 case mappings as per RFC2812 and the support is implemented in IrcString.h. However, these case mappings can vary between servers (many of which just use ascii now). Need a new helper class that parses RPL_ISUPPORT  numerics and provides basic string functionality (e.g. like strcasestr(), strcasecmp()) that respect the case mapping
• Internal ignore list
  The original BNX would permanently ignore users who told it to "shutup" or who tried to abuse it.
• Channel accounting
  Need to maintain internal channel lists which is somewhat more challenging than Battle.net. In IRC, channel information is conveyed on JOIN through RPL_NAMEREPLY. More detailed information (such as complete hostmask) is conveyed through RPL_WHOREPLY. XChat, for example, builds a channel list by first populating the nicknames and then occasionally use WHO to collect host and away information.
• Access system
  Need to implement the BNX access system. This access system would be based on wildcard hostmasks.
• BNX Commands
  The BNX commands need to be implemented (except for designate). Anyone know the difference between BNX's squelch and ignore?
• Configuration files
  Lastly, the configuration files need to be implemented. I think INI would be a good choice.

After I get through these, I'll mark the project as Alpha.

[nslay]

Getting close to finishing a baseline.

I implemented the following

• The access system
• The following commands
  
  • login
  • logout
  • say
    Due to the nature of IRC, this now requires a destination argument (e.g. say #nslay hi).
    
  • chatter
    This also clears the ignore list (the original ignored users indefinitely ... and not by /ignore).
    
  • shutup
• The ignore list
  This just ignores users who say "*shut*up*" (case insensitive). The flood protection isn't implemented yet.
  

[nslay]

I apparently also need a send queue (which is not obvious). I guess it doesn't surprise me.

See here:
http://forums.unrealircd.com/viewtopic.php?f=9&t=7654

I also prepared a test sequence for the original BNX to see which commands could be accessed at which levels. Here's the script:

Code: [Select]

#!/bin/sh

MakeCommands() {
        echo "2010 NAME nslay"
        echo "1007 CHANNEL \"cheese\""
        echo "1001 USER nslay 0012 [CHAT]"
        echo "1002 JOIN test 0010 [CHAT]"

        #set -x
        wine bnx.exe &

        sleep 5

        for i in `seq 0 20`
        do
                level=`expr 100 - ${i} '*' 5`
                user="enslay${i}"

                echo "*** level = ${level}" 1>&2

                echo "*** login" 1>&2
                echo "1004 WHISPER ${user} 0010 \"login passwd\""
                sleep 3

                echo "*** say" 1>&2
                echo "1004 WHISPER ${user} 0010 \"say test\""
                sleep 3

                echo "*** kick" 1>&2
                echo "1004 WHISPER ${user} 0010 \"kick test\""
                sleep 6

                echo "*** ban" 1>&2
                echo "1004 WHISPER ${user} 0010 \"ban test\""
                sleep 6

                echo "*** unban" 1>&2
                echo "1004 WHISPER ${user} 0010 \"unban test\""
                sleep 6

                echo "*** shitadd" 1>&2
                echo "1004 WHISPER ${user} 0010 \"shitadd test\""
                sleep 3

                echo "*** shitdel" 1>&2
                echo "1004 WHISPER ${user} 0010 \"shitdel test\""
                sleep 3

                echo "*** shitlist" 1>&2
                echo "1004 WHISPER ${user} 0010 \"shitlist\""
                sleep 6

                echo "*** join" 1>&2
                echo "1004 WHISPER ${user} 0010 \"join cheese2\""
                sleep 6

                echo "*** join" 1>&2
                echo "1004 WHISPER ${user} 0010 \"join cheese\""
                sleep 6

                echo "*** shutup" 1>&2
                echo "1004 WHISPER ${user} 0010 \"shutup\""
                sleep 3

                echo "*** chatter" 1>&2
                echo "1004 WHISPER ${user} 0010 \"chatter\""
                sleep 3

                echo "*** squelch" 1>&2
                echo "1004 WHISPER ${user} 0010 \"squelch test\""
                sleep 6

                echo "*** unsquelch" 1>&2
                echo "1004 WHISPER ${user} 0010 \"unsquelch test\""
                sleep 6

               echo "*** splatterkick" 1>&2
                echo "1004 WHISPER ${user} 0010 \"splatterkick test\""
                sleep 15

                echo "*** userlist" 1>&2
                echo "1004 WHISPER ${user} 0010 \"userlist test\""
                sleep 90

                echo "*** voteban" 1>&2
                echo "1004 WHISPER ${user} 0010 \"voteban test\""
                sleep 60
        done

        echo "1004 WHISPER enslay 0010 \"login passwd\""
        echo "1004 WHISPER enslay 0010 \"shutdown\""

        wait
}

MakeCommands | nc -l 6112


Nifty what you can do with netcat!

[nslay]

I'd say it's about 85% done. Maybe in at most a couple weeks it will be alpha-quality. I'll port it to Windows, Linux and OS X (if I can find access to one).

You can see the current status here.

The first release is meant to be an accurate recreation of the original BNX. Maybe after that, I can add some extra features.

[nslay]

Good news, IRCBNX is practically done. I implemented ban, unban and splatterkick today (among many commands in the past week). That leaves one remaining command, voteban as well as some BNX-specific behavior.

If you have the time and interest, do have a look over the source (Look under "Browser SVN" on sourceforge) and let me know if there are any design improvements that could be made.

[nslay]

I added voteban support and configuration file support. I ended up having to write my own INI parser since many existing libraries consider # the beginning of a comment. I also ported it to Windows which actually went surprisingly smoothly. Very little had to be modified and most of that little bit was isolated to the IrcClient base class.

I have a single bot (it's configurable for several bots ... not that it really matters) sitting in #ircbnx on irc.freenode.net for testing purposes.

It's not terribly exciting ... but there you have it.

The project is now promoted to alpha status though it still has some remaining missing details and hopefully I can get around to making some Linux, Windows and OS X binaries.

[nslay]

Check this out:
http://forum.valhallalegends.com/index.php/topic,18164.msg185900.html#msg185900

Invert demystified the origins of BNX.

[nslay]

So, I'm ready to make some binaries, though I'm not sure whether I should statically link the libraries or not on Windows. Opinions?

A static ircbnx binary for 64 bit Windows is ~7MB ... kind of clunky. Still, that incorporates PCRE, libevent2 and libstdc++. Though, I like the idea of simply using ircbnx without needing to find runtimes ... and I'm sure any users (if any) would appreciate that too.

Also, I hear Visual Studio's license does not allow open source development ... that you must protect the source code when you distribute the executable. Any truth to this?

Either way, I've been able to produce mingw32/64 compiled binaries. I might try using clang + libcxx instead of mingw and GNU libstdc++ for fun.

Comments?

[Newby]

The lack of dependency fetching is a nice draw to the software, statically linking is cool.  And I'm not sure about the VS license and open source confliction...  (I figure I'd respond since you seem actively into this and nobody else is remarking on it.. I've been reading it without responding for the most part.  It's a really cool project, I just wish I had free time to appreciate it. )

[nslay]

The lack of dependency fetching is a nice draw to the software, statically linking is cool.  And I'm not sure about the VS license and open source confliction...  (I figure I'd respond since you seem actively into this and nobody else is remarking on it.. I've been reading it without responding for the most part.  It's a really cool project, I just wish I had free time to appreciate it. )

The bot is practically finished. Every major feature of BNX (except designate) is implemented (with some extra IRC-specific functionality).

For the past 2 weeks, one instance (nickname is enslay) has been sitting on three networks: binaryninjas (#dojo), betawarz (#dk187,#beta), freenode (#ircbnx).

No problems yet.

Though, I'd like to add server-side ban list management (since IRC severely restricts the number of ban entries) to wrap up the first version.

In future versions, I think I need to embed a scripting engine into it to make it competitive with other bots. It also needs more secure access (e.g. over DCC) since server ops can potentially sniff messages you send through the server (though, the server really ought to be trusted ...).

I'll keep you (the forum) posted.

[nslay]

I released 1.0. It's mostly just tidying up what was there and writing documentation. No server-side ban list management yet. It's much trickier than I anticipated.

I compiled 32 bit and 64 bit (why?) versions with MingW. Everything is statically linked, so hopefully it works correctly out of the box. I'm especially curious if it runs on Windows XP. The "latest" download is the source code, so you'll have to navigate to the "ircbnx" folder to get the executable.

Once I remember how to use rpmbuilder, I'll package it in RPM too ...

In all honesty, I wanted the bot to background like a proper daemon but I'm not sure how to deal with this in Windows (I don't really want it to be a service). So, it'll just show an empty Command Prompt like the old one ...

Feedback and suggestions are appreciated ... if you care.

[rabbit]

Run it windowless with an icon in the system tray (or whatever they are calling it nowadays)?

[nslay]

Run it windowless with an icon in the system tray (or whatever they are calling it nowadays)?

Thanks for the suggestion. As I am largely unfamiliar with Windows API, can you refer (if you know) to an MSDN reference?

[rabbit]

http://msdn.microsoft.com/en-us/library/windows/desktop/bb762159(v=vs.85).aspx

[nslay]

http://msdn.microsoft.com/en-us/library/windows/desktop/bb762159(v=vs.85).aspx

Much appreciated!

Random:

Here's a picture of a Flemish Giant rabbit:

[nslay]

In addition to the basic BNX features, I've recently added (among a lot of internal code cleanups and added standards compliance)

• Channel flood protection
• rejoin command
• who command

I plan to add a seen command too. Any features you'd like to see in IRCBNX?

Why who and seen you ask? Firstly, everyone on most IRC networks I've frequented are automatically invisible (mode +i), so you can't actually make any practical use of IRC's WHO command unless you're in the channel you're querying! Secondly, IRC's WHOWAS is apparently sometimes disabled (e.g. on freenode). I originally thought, "Gee, IRC has WHOWAS, why bother with seen?"

IRC folk apparently have trouble understanding how who is useful. Aside of good debugging information, who allows you to list the users in a channel you may not be able to join to ask for help (for a variety of reasons: banned, channel limit, invite-only, keyed, etc...). And well, a historical Battle.net reason for who was that you could determine if a server was split. I remember those days when you could see users in a channel that no one else in the channel could see and bots' who command was useful for checking if the bot was on a partially split server. The same applies to IRC I think ... IRC has serious splitting problems like early Battle.net (I almost wonder if Battle.net 1.0 was a heavily modified IRCd!).

I'm not so certain I'll embed a scripting engine now (many people seem to find it a PLUS that it doesn't embed a scripting engine!). At this point, I'm looking for simplicity ... I look at eggdrop, for example, and it feels like you have to be an expert to set it up. But it's IRC bot ... who the fuck wants to read documentation to set up something like that? No, Battle.net bots were superior in that respect ... very simple, functional, and entertaining. These IRC guys are crazy!

I'll have to pick Invert's brain about his modified BNX's botnet support (if he remembers) ... is a botnet really useful enough to be troubled to implement it?

I'm also conflicted on SSL support. I think Unreal is the only IRCd that supports SSL and with the export/customs laws, I'm not really sure it's worth the trouble. It'd be pretty easy to setup with OpenSSL though. Comments?

[nslay]

I've added a seen and lastseen command just yesterday. Normally, IRC supports WHOWAS, but it apparently can be disabled by the server (e.g. freenode). seen merely tells you if the bot has seen someone, when and in which channel the user was last seen. The seen list is updated on join and privmsg (to a channel). A timer writes the updated seen list to file once every 5 minutes. lastseen tells you who has been seen in a channel in the past day.

I'll try to address the daemonization problem today. It would be nice to be able to daemonize the bot on both Windows (without being a service) and on UNIX-like systems.

I'll try to release 1.2 in early June.

[nslay]

http://msdn.microsoft.com/en-us/library/windows/desktop/bb762159(v=vs.85).aspx

I've been too busy to work on this lately. I did implement Daemonize() for Unix though, just not Windows. I'll try to tackle this today.

Any suggestions for a tray icon?

EDIT:
I'd like to use the CHAT icon from the old days. I'm just concerned about copyrights ... unless you can find this icon in a collection of Blizzard-released website fan kits?

[nslay]

A distant plan for this project is a learning-based NLP response system. It's been my life long dream to create a bot that can talk with some amount of intelligence ... of course, this wouldn't really be BNX-like anymore (BNX-like IRCBNX is in the 1.0 branch).

It doesn't help that I work purely in computer vision. NLP is a subject I have to teach myself on my own time.

[rabbit]

A rudimentary system is to use a Markov chain and just leave the bot idling in popular channels for a while to build the dictionary.

[nslay]

One step closer to ShellNotify junk. It now runs with WinMain() instead of main() among other feature and code changes.

Latest version is on SVN, but I'll package a 1.2.0 soon.

[nslay]

OK, tray icon is added. It just needs a picture now! Any suggestions?

Also, I'm not a Win32 programmer. Can someone critique these?

BnxWin32Driver.h
BnxWin32Driver.cpp
ircbnx.rc
Resource.h

Thanks.