Tuesday, April 28, 2015

Tamriel Underground

I've been working on a fan site lately; a home for the Thieves and Scoundrels of Elder Scrolls Online to gather and share news.

I present to you, the Tamriel Underground:


Tamriel Underground is a members only site for the Thieves of Tamriel to share tips, tricks, maps and guides.  It will be a place to post Thieving challenges and share your exploits.

Currently only the Maps Tab is active, and I've taken maps from the Rogue's Folio (by reddit users /u/anemonean -- props) and uploaded them and organized them to be refined and searched on.


You can view the maps in a list, and refine them at the top or use the search bar to refine the list down to what you are looking for.  Once you click on the map image you want, it pops up in a lightbox, letting you see the full map.


The website is in it's early phases, and there isn't a lot of content at the moment, but the plan is to create a massive collection of all of the best Thieving tips and tricks and to keep away all those pesky Guardians that will eventually be trying to steal our secrets.

Come join The Underground, where the night is your friend and stealth is your weapon.




Thursday, March 5, 2015

NodeJS - A Little Experiment in Load Testing and Clustering

Load Testing NodeJS on Multiple Cores

Check Out the Git Hub project HERE

Something I wanted to investigate was using Node JS on a multi core system to do CPU intensive applications under high load, so I created a project to do just that.

There are a few ways to use multiple cores in NodeJS, two of which are Cluster (part of the NodeJS API) and WebWorker Threads.

I will preface this by saying I am no NodeJS expert, this project was simply for learning on my part and there may be a better way to go about this with an NGinx set up, but I wanted to share my findings and maybe you'll find it cool too.

The Planning Stages

I started the project out with a simple NodeJS and Express server, which when called with a number in the query, would calculate and return the Fibonacci sequence for that number to simulate high CPU calculations.

My first thought was to use Web Worker Threads to spin up a thread to do work on, and just pass in the data and let it work on a background thread, but this caused issues under high load as too many threads were being spun up (I had no cap at the time) and at the start they were descoping and causing segmentation faults.

I ended up fixing the Seg Fault issue, but even so, they were burning through all of my VMs 2GB of memory and crashing.  

I attempted to create a Worker Pool function to control the amount of threads at any one time, but it quickly got quite complicated.  The idea was to use a queue and simply drop information off and process when ready, but with web requests and waiting on data, this as well got very complicated.

Then I stumbled across the Cluster section of the NodeJS API and found my solution.  With the Cluster API you can spin up multiple instances of your node server on the same port.  So I decided to test if this truly would improve my performance.

Load Testing

I installed loadtest in order to deliver high concurrency testing to my application and began to test the application.  On one branch of my git project I had the vanilla, single threaded NodeJS server which would generate a Fibonacci sequence, and on my other branch I had my project which generated 4 servers on a single port, which would use up the multiple cores.

This was a MUCH simpler solution to building a multi-threaded system within a single event loop and was extremely easy to set up.

So the load testing began.  I called out to my application like so:

loadtest -t 20 -c 32 http://localhost:3030/fib?num=25

I pinged my application using various concurrency values for 20 seconds testing periods.  I tested up from 1 concurrent connection up to 1000 connections.

While I don't believe all four cores were actually being used (as it was on a VM and my computer didn't crash) I did see quite an improvement using the multiple core approach (as one would expect).

The Results

The results from the load test are as follows.  Load test outputs the maximum response times for 50%, 90%, 95%, 99%, and the maximum response time as well as Requests Per Second, Mean response time, and Total Requests.

Computer Specs:  The VM was an Ubuntu 14 VM with 2 GB Ram, 4 processors.

Each request was made with a Fibonacci sequence of 25 for 20 seconds.

NodeJS Single Event Loop

ConcurrencyCompleted Requests50%(ms)90%(ms)95%(ms)99%(ms)Max (ms)Req / SecMean Lat (ms)
648891135169180245286429150
20081103361276135833007364405480
1000817812353726762615726162564092510

NodeJS Multi-Core with Cluster

ConcurrencyCompleted Requests50%(ms)90%(ms)95%(ms)99%(ms)Max (ms)Req / SecMean Lat (ms)
64150097015519332756075080
200144982524765677561062596330
100014923117219542538361876417181350

As you can see the results from using the multi-core approach were much better than the single event loop, and using Cluster is super easy to do.  

You can fork the GitHub project here:  https://github.com/WakeskaterX/NodeThreading

The main fork is (well master too, but) hostedVM which has the standard deployment of nodeJS and express, and the multi-core approach is on the hostedVM_multicore branch.

Feel free to test it as well and let me know how your results are with better machines than my very low powered VM.

Cheers,
WakeskaterX

Tuesday, January 27, 2015

PlayCrafting Boston Winter Expo - February 24th!

Hey All!  Exciting things have been going on the past few months, such as learning new web Technologies and taking part in GGJ 2015 and all sorts of exciting things!   But, this post isn't about that, it's about...

The PlayCrafting Boston Winter Expo!



Stop by at 6pm at the Microsoft NERD center in Cambridge on February 24th, 2015 and check out tons of awesome local indie developers and their games.

You can sign up at the event bright page here.

I'll be there showing off SBX: Invasion and giving away free game related goodies!

Sign up before Feb 23rd to get a discount and come hang out with your local indies!

-Jason C.

Sunday, November 2, 2014

Converting Visitors to Register - adding an Unobtrusive Pop Up

So after about a month or so of having my e-mail login active on my web page, I have exactly zero sign ups.  This isn't unexpected so I'm working on drawing more attention to the login and register buttons at the top.

A lot of websites see great results from using a full page popup when a new visitor visits the site, but generally when I see these I immediately leave the page, and it's fairly infuriating.

So my goals with the most recent pushes to the website were:

1.  To have a pop up that notifies new users about the registration button, but have it be off to the side and non obtrusive when viewing the website and easily closed.

2.  To make the registration process a bit more intuitive and flow a little easier.

For the first goal, I created a little speech bubble pop up that will show only if a user does not have login information saved on their computer, and when it pops up, it will only show for about ten seconds, and won't show back up for 24 hours.  I might extend this to 48 hours / a week, but I don't think users are visiting my site more than once within any length of time.

Here's an example of the pop up:



I wanted it to be mostly out of the way, and while it does cover up some buttons on the UI, it has a clear close button that closes the box immediately.

The second thing on my list was to make the registration process easier.  Before, the login and registration buttons were tied together, meaning you had to click them both and that would pop up the login prompt, then you had to click the register button, and then register.

These buttons have now been split out into a separate login and register button so that users can choose accordingly and not have to go through an extra click.

Just a few small UI updates that I've been meaning to do to convert more visitors into registered guests.  I'll post an update later on the results of the slight UI changes.


Cheers,
Jason C.

Sunday, October 5, 2014

Automated E-mail Authentication In! Also Anomaly Updated.

So I got the automated E-mailing Systems in the website.  When you register, it will generate an e-mail to the submitted e-mail with a link to authenticate the token.

Right now the token is indefinite, but I'll add in a time based token at some point.  I already laid the foundation for the time token in the database & script, but the system is fairly simple.

As you can see here, the activate.php file just takes a token, does some "very basic" checks on html chars / alphanumeric chars and then simply flips a boolean character in the database saying that this e-mail has been authenticated.



It's important to validate e-mails in this way as you want to keep a high sender reputation from your mail server.  Bounced e-mails are no bueno.

Also Anomaly has been updated to have 10 levels.  The only problem is that it's fairly easy right now because of the rule sets.  I need to add more variety to the rules and include multiple rules on the harder levels to mix things up.  Right now this will let me play around with rule sets, what is fun, what is challenging, etc and get some feedback as well.

I need to mix it up a bit so that the Anomaly is slightly better hidden on the larger levels.  Right now it becomes fairly easy to locate the Anomaly with the ruleset "Adjacent".


Please check out Anomaly and let me know what you think!  I'm always looking for feedback on this kind of stuff.

Cheers,
Jason C.

Sunday, September 28, 2014

Login Function Complete!

The Login and Registration functions are now complete!

I spent all of today porting the back end code from Java to PHP.  It took me a decent bit, but I combined some sections of the code as PHP isn't as conducive to OOP as Java, and I didn't need the modularity I had before.

The site uses some client and server side authentication so if you want to try and break it, by all means please try while my database is largely empty!

I ran into a couple bumps deploying the code tonight, the main one being that yet again, my hosting lacks a certain feature.  Namely MySQL Native Driver, so that means the functions I had built to connect to MySQL didn't work so I had to rework them all to work without the driver.

Minor stuff, but still, annoying.  I'll have to take that into account in the future as well.

However, the website and login functions are fairly secure, I still have a bunch more security to add as well, but for now I'm happy with the deployment.

The next step is using the generated internal token to authenticate e-mails by sending automated e-mails so that visitors can Opt-In to the news feed.

Exciting stuff!  I'll start work on that next weekend, so hopefully I can start authenticating e-mails automatically in the registration process.

Cheers and go register at the website!

-Jason C.

Sunday, September 21, 2014

Updated Website Deployed - but... have to rebuild the back end.

Well I deployed the new website to my HostGator web hosting account... ONLY to realize they don't offer Tomcat Apache support for my plan.

And since my back end is built in Java, that pretty much means I need to learn PHP and rebuild my entire back end.  Delightful.

All snarkiness aside, I'm glad to have the front end built out, even if rebuilding the back end is going to be a pain.  There's a lot to see on the web site, with a galactic map on the home page and 5 game pages to check out, feel free to browse and let me know what you think!


So I guess I'll be learning more PHP to get the back end up and running, but at least I know where to go from here.  It probably means I'll have to set up a WAMP stack on my home machine too which I don't currently have installed.

It's 100% my fault for not reading all the details, but I read they had Tomcat support and never bothered to do any kind of testing.

Anyway, at least the back end stuff isn't visible to the user so I can just work on that and deploy it when it's ready.


Cheers!
Jason