RS3 [NXT] OpenNXT - Open-source RS3 NXT framework

Nov 1, 2014
61
27
0
7u6yca7.png

Thanks, [USER]tousuki[/USER], for sending me a logo! I put the old logo in media below.

Welcome to OpenNXT, the first RuneScape 3 server framework focused on everything NXT. After a few years of working on NXT I have decided to help the scene move forward by making a project hopefully unlike any other. One aim for the OpenNXT project is to be completely independent of other tools in the RSPS scene.

OpenNXT aims to update as frequently as possible (without breaking content or hindering server progression) to stay up-to-date with RuneScape.

If you have not checked out my tutorials and resources on NXT, I highly recommend you check them out before reading this post:

Features / Completion
Scroll down below for more information on some of these. I highly recommend you read the entire thread!

Framework
  • Powerful CLI tooling system
  • Dynamic packet definition system
  • Js5 server
  • Built-in HTTP server
  • Built-in proxy server
  • Lobby
  • World login
  • Region updating (Rebuild normal)
  • Region updating (Rebuild dynamic)
  • Player info packet
  • Npc info packet
  • Anything content related.
Once the framework is at a point where content can be worked on, I will update the list above.

Tooling
  • Client downloader Code
  • RSA key generator Code
  • Client/Launcher patcher Code
  • Cache downloader Code
  • Cache editor
  • NPC spawns dumper

Other
  • Tutorial: "How to update OpenNXT to latest version"
  • Tutorial: "How to identify packet data structures in the NXT client"
  • Creating Github Wiki and and mirroring my tutorials

Features - In-depth
This list might not contain all features OpenNXT has to offer. If something is missing please let me know so I can document it.

Powerful CLI tooling system
Gone are the days of 50 Main files. Gone are the days of weird config values being hardcoded in the server. Gone are the days where you need 10 different Intellij tabs open to develop one RSPS.

OpenNXT has one Gradle project with one main, and everything is bundled in it. You only need one Intellij window open to get started with OpenNXT. From client downloading and cache downloading to dumping anything from RuneScape, everything is accessible through one main.

To achieve this, a mixture of classpath scanning (To detect tools) and Clikt (A CLI library) are used. Simply run "./server --help" for command usage! It's that simple.

Dynamic packet definition system
OpenNXT has a super-simple packet definition system, allowing you to quickly and easily upgrade or downgrade versions without changing a lot of the source code. Every packet supporting this system has a simple text file in the repository with the packet structure written in plaintext. It's really as simple as that. In the future, these definitions will be hot-reloadable, to support even more awesome features.

Here is an example of the VARP_LARGE packet definition in 918:
Code:
id ushortle
value intle

Js5 server
The built-in Js5 server and cache library automatically handle the following for you:
  • Prefetch table generation (aka "Grab" keys). There's sadly a small issue in one of the prefetches that causes the client to display an "incomplete" cache state. Of course I will fix this. Soon TM (meaning: never).
  • Checksum table generation and encryption.
  • Serving the music index and checksum table over HTTP.

Built-in HTTP server
The built-in http server serves all client files (jav_config.ws and binaries) as well as the js5 protocol. No need to mess around with Apache/Nginx and running into http headers or content type issues.

Built-in proxy server
This is the, in my opinion, main selling point of OpenNXT.

As most people involved in the RS3 scene know, the Java client has unfortunately been discontinued. Many people used this to dump data from RuneScape (eg. adding System.out.println to all packet decoders, then playing the game). You can't really do this with NXT. The only option would be memory reading or injecting hooks, but that's all very difficult. To avoid the RS3 scene coming to a grinding stop, I have implemented a very cool feature, which is the proxy.

The proxy is essentially a headless RuneScape client. When you start the server with the "--enable-proxy-support" flag, the server will enable everything it needs to proxy your local connection to RuneScape. If your login meets the requirements for a proxy connection (I'm going to add an username whitelist in the future), the server will open a connection to RuneScape in the background. If the connection to RuneScape succeeds, the server will pass packets on from your client to RuneScape's live servers (and vice-versa). Packets that can be decoded can then be handled to dump data from RuneScape. This allows you to play the game now, and when you want to work on your RSPS, you can implement anything you did on real RuneScape!

Currently the proxy only functions in the lobby.

Long story short: You connect to server. Server connects to RuneScape. Server sniffs data between your client and RuneScape. You can use this data to make new content.

Spoiler for Proxy lobby media / proof:

unknown.png



Client downloader / patcher & RSA key generator
Client downloader downloads all of the clients for the current game version.
RSA key generator generates RSA keys you can use. You only have to do this once. If you re-run this, old launchers won't work with newly-patched files.
Client patcher patches the clients and launcher based on the RSA keys generated by the tool above.

Cache downloader
Simply downloads the cache, but it's multithreaded and has a few big bugs such as randomly spitting out a billion errors. Have fun fixing it, I just restarted every time it happened, and it worked fine.

Tbh I don't know if it's faster than RSCD, but I managed to cap my internet connection (250mbps) with 3 js5 threads and 8 http workers, I was bottlenecked on disk I/O. But it's bundled so hey, can't complain right!

Other
I might add/remove/edit the above section to reflect the features that stand out the most.

Source code
The source code is fully licensed under GPLv3 and can be found on my GitHub profile. Fork, watch, star and PR away!

Credits
Without the contributions
  • All contributors on the GitHub repository, for their obvious contributions to the project.
  • Everyone who motivated me to work on RSPS and NXT or who I talked with about RSPS. While I cannot remember everyone, here's a few: [USER]velocity[/USER], Log, Daron, [USER]woahscam[/USER], Ceikry, Kermit, Summer, [USER]ARMAR X K1NG[/USER], [USER]freeezr[/USER], [USER]Tommeh[/USER], [USER]Displee[/USER] and more
  • Cjay0091 for Matrix. While OpenNXT is written from scratch, it does/will contain code inspired by Matrix.
  • Displee for his cache library. While OpenNXT's cache library is written by me, I took some inspiration from Displee's cache library.
  • Velocity, Cube and all other contributors of the Great NXT Beta Dump Thread


Media
There is not a lot of media yet because essentially it's a bunch of framework and tools now.

Spoiler for Media - more will be added in the future:

World login!
unknown.png


Old logo since people liked it
unknown.png

i need a project logo
OpenNXT - Open-source RS3 NXT framework
Moving RS3 RSPS forward one commit at a time

NXT client boots to login screen using only built-in tools:
unknown.png



Small preview of the tools CLI
unknown.png



Proxy intercepting lobby packets and dumping them to the console
unknown.png

 
  • Like
Reactions: Tamatea and Graham
Good luck! :D:D

Thanks!

Best of luck. Hopefully, people contribute to this rather than try to take advantage of it.

Thanks!

I think people will take advantage regardless. I just hope that people contribute one way or another if they use anything from this project though. I released this since I don't see myself hosting a RSPS, but I know a thing or two about NXT. Might as well see if this can move the RS3 scene forward a bit :)
 
bump, simply because it hasn't had the credit it deserves yet
 
Interesting approach to packet decoding. Unless I missed something, the system could be more fleshed out to actually generate the boilerplate code for the codecs rather than having this weird map shenanigans. Saifix had done a project similar awhile back and there was another project in here that was messing with protobuf.
 
bump, simply because it hasn't had the credit it deserves yet
Thanks :)

Finally an innovative project worth following.
Yeah. I've been very busy with work but still intend on going on with this project. So updates might come a bit slow :(

Interesting approach to packet decoding. Unless I missed something, the system could be more fleshed out to actually generate the boilerplate code for the codecs rather than having this weird map shenanigans. Saifix had done a project similar awhile back and there was another project in here that was messing with protobuf.
I think it's possible to generate the bytecode automatically based on the packet definitions. However, since I wanted to get a working prototype as soon as possible, I just went with what worked. The advantage of this approach is that you don't have to write both the encoder and decoder for every packet, you just have to write the packet structure and it's done.
 
Without a doubt one of the best RSPS devs out there :)

Good luck!

Thank you!

I have just updated most of the framework to #919. Logging in to OpenNXT's lobby works for the most part. Logging in to both RuneScape's lobby and game through the proxy seems completely functional as well.

Notable changes:
- New js5 index
- New prefetch (I assumed for the new js5 index)
 
  • Like
Reactions: Dyrroth
Thank you!

I have just updated most of the framework to #919. Logging in to OpenNXT's lobby works for the most part. Logging in to both RuneScape's lobby and game through the proxy seems completely functional as well.

Notable changes:
- New js5 index
- New prefetch (I assumed for the new js5 index)

thanks for this contribution
 
I don't understand it that well, Im not sure how to implement it into my 876 server, i have 910 cache but I am not that skilled at setting this up for my own testing server
 
I don't understand it that well, Im not sure how to implement it into my 876 server, i have 910 cache but I am not that skilled at setting this up for my own testing server

Hey, this is a standalone framework/base, so not something you can really implement in another framework. You'd have to look at the networking and internals to port it over to Matrix3 or VirtueRS3. Unless you really know what you are doing, I wouldn't recommend taking that approach though.

Of course you would be able to use the tools (such as the client downloader, client patcher and the proxy)
 
Added world login. It's incredibly hacky, but I wanted to see if things didn't burn to the ground in the process. Clean-up will follow soon. Or never. We'll never know I guess.

Player updating doesn't work yet and the location in the packet is hardcoded, but can easily be changed.

unknown.png

No, I am not activating Windows
 
Added world login. It's incredibly hacky, but I wanted to see if things didn't burn to the ground in the process. Clean-up will follow soon. Or never. We'll never know I guess.

Player updating doesn't work yet and the location in the packet is hardcoded, but can easily be changed.

unknown.png

No, I am not activating Windows

Very pretty :pika:!
 
Added world login. It's incredibly hacky, but I wanted to see if things didn't burn to the ground in the process. Clean-up will follow soon. Or never. We'll never know I guess.

Player updating doesn't work yet and the location in the packet is hardcoded, but can easily be changed.

unknown.png

No, I am not activating Windows
this is fkin amaxing dude
 
Hey, I was wondering, if you were to start implementing content would you use scripts which the engine could run or would you go with the more widely-used approach of adding content directly into the engine? If you decided to go the scripting route would you; pickup an existing scripting language such as lua or lisp or make something resembling runescript and compile that to kotlin? Really cool project, all your resources are always very helpful.
 
Last edited:

Users who are viewing this thread (total: 1, members: 0, guests: 1)