More questioning thoughts

 

So the question is: what kind of content should I produce?  What do I know that other people don't know?  

What do I know that I can teach to other people? 

I spend most of my time trying to understand the choices I have made, and examining systems that will make better choices the next time.  Part of that is habit creation, and part of it is analysis.   Perhaps the real problem is that I haven't spent decent time in actual analysis of behaviors and choices.  

So what is the sufficient amount of time?  Right now, I go by the seat of my pants on making choices.  Bad experience compounded a lot o times to create answers that are generally correct.  But then generally correct is not always correct.

And there goes the perfectionist mentality in me, trying to find the perfect solution.  Whereas the trained brain portion says pick any solution.  So where is the distinction?  Probably in the question: does this decision matter in the next two years?  If its the difference between the chicken and the fish, then the decision won't matter twenty minutes from now.  

But aren't many of our decisions like that?  Many of them are very small and insignificant in their own right.  I've always been a skinny guy, and it doesn't matter what I eat.  So I don't have the problem that others have with weight.

On the other hand, I have a problem with weight.  I'd much rather add another 30 pounds of muscle.  Would I be happy if I did that?  I don't know.  That depends on whether I'm looking forward to where I want to be, or looking backwards to where I was.  

I heard a pretty good description of the subject form Dan Sullivan.  He described it like thus:  at the beginning of the period, you are a position A.  At the end, you are at position B.  Where you want to be is position C.  

Whether you are happy or not is what you do when standing on position B.  Do you look at C, and see how far you have to go?   Or do you look back at A and examine how far you have come?  The happy person stands at B and looks back towards A.  The unhappy person looks at C.  

It's all a question of expectation vs reality.  Did you always compare yourself to the expectation?  If so, you are comparing yourself to something you can never be.  Despite all your hard work, you will never meet that reality.  That image of what you want to be is something you are highly unlikely to ever reach.  Partly because the ideal is far away.

So how do you end up cultivating that on a regular basis?  And what makes us think that it's a good idea to avoid reality with such measures?  A lot of both probably comes from what we have been inadvertently taught wrong.  Gratitude is rarely taught in class, and it isn't something you see glorified on TV.  Telling people the way it is seems to be the main method of the day, though that has a tendency to aim and nothing and hit it every time.  

Perhaps the problem is not in either approach.  It's in not producing.  Realistically, the difference between success and failure is not gratitude or any other routine.  It is about producing versus consuming.  The more you produce, the more you grow.  You could think of production in the agricultural sense.  Production is planting seed that may or may not grow.  The more seed you plant, the more likely you are to get a giant crop. 

But most people plant no seed, and wonder why they don't get a crop.  Out of the 24 hours in a day, they spend 0 of them producing.  Of the things they do produce, they are producing for someone else.  But producing some product, no matter how bad, for ourselves?  Never.  Can't possibly do that.  

Success is not about what you want, it's about what you do.  And if you want to break out of what everyone else is doing, then you must create content.  It doesn't have to be original.  Let's face it: Motzart's first symphonies weren't original.  The beginnings never are.  But they are just those: beginnings.  You have to start and go somewhere or you will never end up anywhere.  

Upgrades break your stuff.... python 3.4 vs python 3.7

So...  The new version of PRTG rolled out and Python 3.4 is now deprecated and Python 3.7 is a thing. 

What does that mean for me? 

Everything I wrote before is now broke....

so...  that big chunk about paepy...  you can forget all that.

Luckily, I spent the time figuring out how to make all this work.


starting point...

result = []

result is going to hold our collection of strings that we'll add together prior to dumping into PRTG.


Next bit.. 

We need this bit to create the proper JSON format before we start dumping stuff in. 

 result.append(" { \"prtg\" :{ \"text\": \"OK\", \"result\":[")


Second bit... 
result.append("{\"channel\": \"name of the thingy\", \"unit\": \"Custom\", \"value\": " + value_of_importance + ", \"is_float\": \"True\"  , \"primary_channel\": \"True\", \"warning\":\"0\", \"is_limit_mode\": \"True\", \"limit_min_error\":\"0.5\", \"limit_max_error\":\"1.5\", \"limit_error_msg\":\"the message\"}")

So... you're going to have to escape a lot of quotes.  General stuff....  escaping quotes is \".   So if you want a quote (")  you need to write \" to get a quote and not finish the line.

It's a bit pain to say the least.

The not so much pain is that the end result can be dumped into a JSON validator and you can find what needs to be fixed and what is wrong.

I like this one.  https://jsonformatter.curiousconcept.com/

Other web pages I found valuable:

  https://kb.paessler.com/en/topic/80232-how-can-i-monitor-sigfox-callbacks-with-prtg#reply-276903

https://www.paessler.com/manuals/prtg/custom_sensors

Please use the previous article I wrote on troubleshooting to find the actual JSON output and dump that into a validator to make sure your JSON is good before pulling your hair out.  The big thing that is going to be a problem is adding the correct amount of commas after the channels.  

One comma at the end of each channel, except for the last one.

Use the len() command to solve that problem.

To be more specific.....   len() = blah.

Create another variable...

currentRun = 0

Increment variable at the end of the for statement.

if len() > currentRun:

    result.append(",")    #add a a comma

That should be sufficient...

Now....   One everything is done...  

Need to add this bit to the result so we can the end of the JSON.

result.append("]}}")

Ok... got that...  

Now everything is is stuffed into result, but we need to get rid of the commas that normally show up when you just add a bunch of strings together.....

So...

finalResult = ""

         for items in result:

             finalResult += items

    

print(finalResult)

And with that print statement, we're done.

We've got rid of commas in the wrong place, got rid of paepy, and upgraded from Python 3.4 to Python 3.7.....

Looks like we're good.

PRTG: solving your problems

So you wrote a nice custom script to monitor you Next Cool Thing, and it doesn't work.  Not only does it not work, you can't figure out what is wrong with it.

So how do you solve it?

First, turn on sensor debugging. 

Go to the sensor in question, then Settings.  Scroll down until you find this bit. 

Turn that on.

Got that?   Good.

Now the interesting part.  Where are my readings? 

C:\program data\Paessler\PRTG Network Monitor\Logs (Sensors)\

So there you have it.

Kind of.

You get two files for every sensor.   One file indicates all the stuff that was sent to the sensor via PRTG.  That's the file with the name "Result of Sensor XXXX.Data.txt"

The other file is the output after the script has been run.  That's "Result of Sensor XXXX.log"

From there, it's time to go read that second log file and try to figure out what the problem might be.  The big problem I have with troubleshooting Python scripts is the lack of an Idle interpreter in the default installation of PRTG. 

That's survivable.  Notepad is there.  It's just more annoying. 

The second part of reference is that the interesting bits about what failed is generally at the very end of the script output. 

Is this the perfect debugging scenario?  No.  But it does provide the information you need to figure out why your script isn't working.

You did write the script on a machine with better debugging utilities to try and make a good proof of concept, right? 

So best practice in my eyes: write the entire script except the output bits on a separate machine before transferring it to the remote probe and/or primary server.

So...   Have fun, and go squash some bugs.

PRTG: Making Custom Sensors to monitor strange things with Python

#verion 1.0

#last modified 12/2/19

#

#v 1.0   initial revision / prtg integration

#

import sys

import json

import urllib.request

from paepy.ChannelDefinition import CustomSensorResult

if __name__ == "__main__":

    location = json.loads(sys.argv[1])

    

    parsed = "http://" + str(location['host']) + "whatever else in the url"

    

    page = urllib.request.urlopen(parsed).read()

    

    #data comes out as binary type.

    #convert from binary to normal string

    np = page.decode('utf-8')

    scrip = np.split('

    for lines in scrip:

    result = CustomSensorResult("OK")

         result.add_channel(channel_name="channel1", unit="Custom", value=status_value1, is_float=True, primary_channel=True, warning=0, is_limit_mode=True, limit_min_error=0.5, limit_max_error=1.5, limit_error_msg="channel1 failed") 

         result.add_channel(channel_name="channel2", unit="Custom", value=status_value2, is_float=True, is_limit_mode=True, warning=0, limit_min_error=0.5, limit_max_error=1.5, limit_error_msg="channel2 failed")

    print(result.get_json_result())

Base code for a python script sensor for PRTG.   

Using this method, you write a script to scrape a web page, and then present the information to PRTG as a JSON file.

The limits are used to define up/down/warning status.

In this case, I have outputs of a binary 1/0 for working not working.

So how does that work?   

Notice this bit.

result.add_channel(channel_name="channel1", unit="Custom", value=status_value1, is_float=True, primary_channel=True, warning=0, is_limit_mode=True, limit_min_error=0.5, limit_max_error=1.5, limit_error_msg="primary connection 

And breaking that apart, the section here

is_limit_mode=True, limit_min_error=0.5, limit_max_error=1.5, limit_error_msg="channel1 offline"

Let's break these down.

Note: these names aren't the same those expected or presented for EXE/Advanced sensors on the PRTG custom sensor page.

is_limit_mode = Sets PRTG to know that the output has acceptable ranges of input.   

limit_min_error  = This sets the lower limit that defines an error.  Depending on your output, there may not be one.  I'm using outputs of binary 0/1 in this case, so I set it to .5.   Therefore, a 0 output is defined as error state.

limit_max_error  = This set the upper limit that defines an error.   Depending on your output, there may not be one.  In my case, there is never an upper maximum error.   So I set it to 1.5.   

limit_erro_msg = The message you want on PRTG for any device that may be not working.

So, with these setting set correctly, PRTG will report an individual sensor is down based on the values you assign to status_value1 and status_value2.  So now, you can alert based on those settings using normal PRTG alerting.

What this base script doesn't currently do:

1. Parse anything.  Parsing the web page is based entirely on what you are looking for.  The page I was looking at was all table based, so splitting the data into tables made sense.  You will have to handle that portion.
2. Deal gracefully with urlib.urlrequest.open() errors.   You will get a JSON error in PRTG when you try to pull a web page you can't get.  That's a simple try/except statement.  Use this message in your except portion to report failure gracefully.

            result.add_error("Your Error Message Here")

Secondary important thing...   probably the most important.  

See this block? 

 location = json.loads(sys.argv[1])

    

    parsed = "http://" + str(location['host']) + "whatever else in the url"

This block accepts json data as input to the script.  

The second part location['host'] pulls the IP address setup on the sensor to feed that data into the script.   So this script can be written once and run on multiple devices.  That's what makes this script extendable.

Now...   

So, you've got the initial script working.

How in the world do I troubleshoot this thing when I suddenly get a bunch JSON errors when I deploy it?   

That's the subject of another discussion.

Thoughts in consideration

Judge success not by reaping but by sowing.  

Focus on something for 30 minutes and think of nothing but that thing.

Creating an iron will based on creating and following rigid systems 

Hard work isn't the answer.  Impact wasn't improving.  Figure out what works.

-What am I required to do?

-What gives the greatest return?

-What gives me the greatest reward?

Focus your attention on (1) what you must do, (2) what you ought to do, and (3) what you really want to do.

Subjects I need to study and understand:

[]Restraint

[]Generosity

[]Hard work

[]Motives

[]Honesty

[]Patience

[]Humility

[]Productivity

[]Dependability

[]Temper

[]Attitude

[]Facts

[]Goals

[]Planning

[]Boundaries

[]Proper Thinking

[]Common Sense

[]Prosperity

[]Emotions

[]Sowing

[]Direction

[]Correction

[]Conflict

[]Pressure

[]Criticism

[]Judement

[]Confrontation

[]Forgiveness

[]Ethics

[]Ownership

[]Ambition

[]Listening

[]Co-Signing

[]Responsibility

[]Debt

[]Saving

[]Developing People

[]Understanding People

[]Inspiration

[]Influnence

Measurement leads to success.  If you can't measure it, you can't control it (notes)

Effective system include application (notes)

What looks like a people problem is often a situation problem (notes)

To change, you have to change the persons situation

self-control is an exhaustible resource

What looks like laziness is often exhaustion

What looks like resistance is often a lack of clarity.

To change you must provide crystal clear direction.

Mis-allocation in abundance leads to scarcity. (notes)

Scarcity requires being right all the time and the consequences are devastating.

Pain and pleasure are the driving factors of all change

Links to all notes are private.  Should the notes become interesting, those will be posted at some other point. 

monitoring applications

Part of a decent monitoring scheme is making sure business critical things are running all the time.  PRTG and powershell are a great way to do that.

$errorFound = $false


try
{
    if( -not (get-process -ProcessName "blah" -ErrorAction 'silentlycontinue' ))
    {
        throw 
    }
    $textVal = "OK"
    $errorFound = $false
}
catch
{
    $errorFound  = $true
    $textVal = "blah Not Running"
}



write-output ""
write-output ""

write-output "blahProcessRunning"


"Status"
   

"1"
"1"
"
"     



if($errorFound)
{
    write-output "0"
    write-output "1"
    $textVal = "blah Process not running." 

}
else
{
        "0"
        $textVal = "OK"
        "1"
    
}

"" + $textVal + ""
write-output ""
write-output ""
write-output ""

write-output "BlahProcessRunning"


"Status"
   

"1"
"1"
"
"     



if($errorFound)
{
    write-output "0"
    write-output "1"
    $textVal = "blah Process not running." 

}
else
{
        "0"
        $textVal = "OK"
        "1"
    
}

"" + $textVal + ""
write-output "
"

Note: Blogger as far as I know is doesn't have a good way to post full code.  So there's a bit more to it than this, but it's all opening and closing brackets. 

Ok, so there's all the code.  Most of it is pretty dull.

The heavy lifting is done by the get-process ProcessName "blah" section.  Which is pretty self explanatory.  Works great when the process happens to be running.  The problem is what happens when the process doesn't run.

When it doesn't run, you get red powershell text.  Which is what we don't want. Adding -erroraction "silentlycontinue" gets rid of the error text.

But now it fails and doesn't tell us anything.  Also not what we want.

So try-catch.

Which doesn't quite properly work.  It doesn't give any output at all.

The answer was the if(-not ((process)) ) bit followed by the {throw}

So, if you get some sort of failure, silently continue and throw an error.
Which the catch picks up.

And now we've got some nicely formatted text that changes based on whether the process is or is not running.  And with business critical processes, that's what you need.

Now, get-process dumps a lot of interesting data that you could feed back to PRTG.  Frankly, I wasn't that concerned.  I just wanted to know whether the process was running.

As for install procedure, let's not skip that.

Install a PRTG remote probe.
Drop this file in the CustomSensors\EXEXML directory.

Change the policy for the x86 version of powershell to unrestricted.
Or sign the code with your own code signing cert.  I don't know how to do that.  So  I go with the first.

That's in c:\windows\syswow64\windowspowershell\1.0\powershell.exe

Not the same one as opening a command prompt or searching for powershell.exe.

So there you go.  Alert as your company dictates and or want.

Thoughts on the c-store

So as working in a c-store is vastly more complex than you'd think, I guess it's time to cover the technological fun of dealing with what's in a c-store.

The primary things everyone thinks of is the point of sale.  Point of sale is an important point, but not the only thing.  In general, most point of sale systems end up having a server, and 1-2 terminals.  On top of that, you have some sort of device to handle the fuel controller.  Sometimes those are IP based, sometimes not.

In the modern arena, you also end up with network connected pin pads...   If we were making a list, we'd have..
1 )POS router
2) POS switch
3) POS controller
4) POS terminals 1
5) POS pin pad 1
6) POS terminal 2
7) POS pin pad 2
8) Fuel Controller

I'd always recommend 2 POS terminals.  Why?  Because if one goes down, all your in store sales are down.  And most of the money made in a C-Store is not out at the gas pump.  The gas pump is just a driver to get the customer inside the store.  In any situation, 2 is 1 and 1 is none.  Redundancy, redundancy, redundancy.

So, we've got the money maker down and out.  Problem is, you still don't have a primary network connection for all the rest of the stuff.

Now, people have tried to merge the POS router into the site level router.  That's a bad idea.  The purpose of the point of sale router is to segment all the money making from the rest of the system.  It's not good to try and merge all that together to save a marginal amount of money.  Don't scrimp on the main things.  Get a good router/firewall and probably the vendor specific point of sale router.

While we're talking about site level routers, you need a backup credit card solution.  I recommend cellular.  Phone dial-up has grown worse and worse, and is generally unsupported decently.  Cellular signal is generally pretty okay, and running a small 300 MB data plan just for backup credit card traffic is well worth the cost.

The cell backup might run you $100 a month.   Realize in one network outage that lasted 3 hours, a large size store can run $500 in credit card traffic.  One outage more than pays for the cell backup for the next several months.

So, we're also going to need a site level switch.  That's to plug all those other devices in that have nothing to do with processing money.  It's all in the automation and scaling.

9) Site level router
10) Cellular backup device
11) Site level switch
12) Site broadband connection

I glossed over the site broadband connection because that generally depends on what you've got in the location.  Large cities will have a great cable modem system.  Small ones you deal with what you can get.  Your needs dictate.  Other than AT&T DSL, I've never had any real issues.  All generally run the same tech and have the same stuff.

My problem with AT&T DSL narrowed down to the inability of them to provide a static IP with unfiltered network to form VPNs.  It's home service for home users.  Anyways...

Now that we've got the primary network setup and the POS running, we've got a system that will work. 

Now all the other things in the store that have nothing to do with making money, but do.

13) Alarm system.
Assuming your store is going to close, you're going to need one.  Most of the one's I've dealt with borrow a port on your network so they don't have to pay for their own.  Cuts your cost a bit.

14) HVAC system
The modern HVAC system is nice and network connected.  So the maintenance team can get alerted when those pesky coolers fail.  And that gets expensive when you lose all that product.

15) Remote power systems (multiple)
I'm primarily talking about devices made by Digital Loggers, Inc.  They are a great company, and make a great product.  So what's the purpose of this thing, and why do you put one (or more) in your locations? 

Ever had to drive several hours just to pull the plug on a device?  Ever hated that?  Ever wanted to do it from the comfort of your desk?

Buy one of these things.  Or more than one.  I put one on every single terminal out of town.  Rebooting a device once generally pays for the travel/time cost of one of these devices.

16) Back office computer
This might be the only traditional computer on site.  Purposes are handling paperwork and ordering.

17) Side bar devices
These are all those stupid little terminals to implement things the point of sale doesn't.  Checks processing, food stamps, money orders, and gift cards.  They junk up the counters and make small amounts of money.  But everyone has them.

18) Printer
A good multi-function network printer is well worth it.  Outsource this sucker as best as possible, and save your IT.  Printers are a mess.

19) NVR/DVR
The device I don't like the most.  Probably because of the expectation of what the DVR is good for.  The purpose of the DVR is for after event examination.  It's not for stopping some kid stealing candy.  You are not going to stop that kid by staring at a tiny screen.

20) DVR POE switch
To power all those cameras that cover everything from the fuel island to the junk isle.

21) Tank Level Sensor
Used to monitor the underground fuel tanks.

22) ATM
In the c-store world, you need to be in the ATM business.  It's not complex, it's not expensive, and it makes decent money.  Throw it on cell backup, and then when the rest of the world is down you can still have people pulling their money out at $2-4 dollars a hit.  Fill the ATM with $20s out of your deposit, and you don't even have to pay separately to fill the ATM.  It's win-win-win. 

23) Battery backup
You need a network connected battery backup.  It doesn't do any good to have a battery backup on your major system and then not even monitor it.  Crazy.  But that's the mentality of many people.  We'll go into that one on a later date. 

24) In store music
Those blaring, pre-built playlists interspersed with your own custom ads make for interesting stuff.  It can either be great or horrible.  It also lets you control the type of music that is played in your store.  Don't like country? Then you'll probably hire an employee that brings a boom box and plays it way too loud.  The same can be said for any type of music or talk radio.  Pay for in store music and control the experience.  It's probably not going to make you much money, but it will give you a unified experience across your chain.

25) In store media
Paper menus are out-dated.  You can do it all with a raspberry pi, piSignage, and a TV.  It's also dirt cheap in comparison to some of the other options. 

26) Gas price signs
These generally don't have a network connection, and integrate directly with the point of sale.  But it's another piece of electronic that needs to be considered. 

27) Safe
I hate safes.  I really do.  I don't think they should be managed by IT departments.  But I do, and I hate them.  They are a necessary evil of the system.  It's the best way to store the money prior to pickup by armored car or before it goes to the bank.  Even if the IT department doesn't manage the device, it's likely the thing has a network jack.  Which means you need to network it and fight with the safe tech that doesn't know the difference between DHCP and DNS. 

Come to think of it, that's most point of sale techs as well. 

Anyways.

That's a separate argument.

So I think that about covers what is in the normal c-store.  Realize that's 27 devices, but could easily be 35-40 without IP cameras. 

How?  I put at least 4 WebPower switches in every out of town location.  So there's 30.  Add another few side bar devices (checks, money order, food stamp, gift card) and you're up to 35 and quickly.

Add the IP cameras, and you've got 60+ network connected devices in one location. 

That's a lot of network for a place that sells cigarettes, beer, and gas.