Ubuntu changes

 I ran an Ubuntu 12 box for a long time.  It was running off some old, cast off hardware.  It was great to mess around with.  For a while, I ran all my accounting through that.  Which had its own problems once my bank no longer allowed a Quicken style exports.  They just allowed pure export into Quicken.  But that's a different story.

Eventually, the computer died.  It was old, and running off cast-off hardware as I mentioned.  So I found some new cast-off hardware, and rebuilt the system.  Except I decided to put in Ubuntu 20.  I'm not sure I like it, but it's here.  There were some interesting download that I found through the Ubuntu store(?) or whatever it was called.  

I realize it's been a while, but the things I liked aren't there anymore.  I guess I spent too much time when I should have been hurrying.  But I digress.  I think it's time to actually spend some time on this.  It's the problem of having too many ideas running through your brain and not enough time to work on all of those projects.  Maybe I need less sleep.  Sometime that works, but only for a short while.

Back to the new Ubuntu store thingy.  It doesn't work as well as it should.  It runs through an icon-based format.  That was something I remember from before.  But this time, the setup doesn't work as well.  Especially when you get to the bottom of the page.  Maybe some updates will help.  I doubt it.  

Anyone know what those bottom two icons are?  I don't.  And I can't read them.  Nor is there a scroll bar to look at them correctly.  And it's not just the entertainment section that has this problem.  All of them have the same design flaw.  I'm guessing whoever tested this didn't spend enough time testing. So let's hope they fix this on a patch.  I've got an update running in the background so we'll see.  

Moving from 12 to 20 is quite a bit of a jump.  I ran 14 and 15 for a bit, but those were all work environments where I was doing specific things and couldn't just play around.  It was all Graylog if you want to look back and examine those posts.  

Let's see what happens when I play more.  

 

Network Security on the Cheap, part 2: Graylog

So, I've spent the last couple of weeks setting up GrayLog in a cluster.

Giant pain in the butt.

My build was Ubuntu 16.04.  And that works fine.

Lots of stuff that is not mentioned, or even thought about.

A few things I had to solve...

1) Elasticsearch needs to be on the same version
2) Mongodb needs to be at least version 3 or higher.
3) The web URI information is badly documented.
4) A cluster won't form until you are using the same database for all servers.

I set up a router in front of my setup, resulting in a NAT'd public IP address and the cluster behind.  Gives plenty of room for growth without ever readdressing anything externally.

Based on recommendations, I changed the listening port to 12900.   Not sure the implications or reasoning of that.  But it seems to work for me.

So, private IP is IP of the server.
Public IP is the NAT'd IP of the server. 

rest_listen_uri =  private ip/api/

web_listen_uri = private ip

web_endpoint_uri = public ip/api/

rest_transport_uri = private ip/api

forgetting the /api/ on the web_endpoint_uri causes a specific issue

It's the cannot POST issue.

Don't forget the /api/ setting.

Once that is done

I also load balanced the web interface using HAProxy.

Final thing about Graylog, and one I wondered about...

Once I've got the cluster up and working, do I need to load balance my UDP syslog traffic?

Short answer: probably not.

Though it doesn't look like the other servers in the cluster are doing anything, they are.  Easy way to tell is look at the number of messages waiting and processing.  With 2 servers in my cluster, my waiting and processing at 4,000 messages per minute sits in the dozens.  There's never a queue.

So, with two low power computers and a whopping total of 14 GB of RAM, I process 4,000 messages a minute with zero wait.  That includes bursts of up to 6,000 messages per minute.

I'm not saying you won't need to load balance the UDP traffic.  I'm saying you need to base it off your scenario.  And figure out how many messages per minute you are getting.

Generally, UDP is low overhead network traffic anyways.  The general problem is not that you can't receive enough data.  It's that the data takes too long to process, resulting in a queue.  That's the purpose of building a cluster.

Total cost at this point?  3 cast off computers that aren't good for much else.  One of my computers runs 6 GB of RAM and runs 100% CPU usage.   Yet it still handles enough to reduce the load of the primary computer.