How to Egress Data from a Network

TL;DR Corporate networks have various SAG’s (Secure Access Gateways) that filter traffic in a way that is constantly evolving to different threats. To bypass most of these a custom “protocol” must be adopted, but not be too extraordinary. Using an HTTP POST is the easiest method to consistently get data off of an enterprise network that you do not control.

For a pointless former side project (aka non-monetizable), one of the DIY webcam I set up at work (for the view on the 14th floor) required a ridiculous amount of work for something that should be very simple. While the exercise of making it work was frustrating, it helped me learn more about how an enterprise network functions and what it takes to get a simple picture off of the network at work.

Previously I was using the AWS command-line tool to transfer images to an S3 bucket, this was dead simple on a home network. But once I got the camera set up at work, the images were not making it to the S3 bucket with all behavior pointing to the network. Mind you this is the Free Wifi at work, so it makes sense that there are more restrictions in place. Rather than suffering defeat, I went about researching how these secure proxy gateways work to filter and identify potentially suspicious traffic.

The best I could tell, any traffic that was not normal web traffic was being blocked or would be shortly after working the first few times. Since I am a frontend engineer I realized that I needed to stick to my domain and focus on building a way to transmit the image using the HTTPS POST method. My thinking is that a POST request is so basic and common that it would be hard to block in an automated manner, which turned out to be correct. For the backend, I chose to use an AWS Lambda function to process the image from base-64 to an image on S3.

Getting the AWS function working using a serverless framework seemed like overkill. Writing the code was all groping via the online editor and running the commands on my laptop. The biggest issue I encountered was the way the base-64 encoding was happening on a Mac vs Linux. Apple really should work to be more in line with the *nix’s, but the easier solution is to write all code on a Linux machine.

My intentions for this project were to have better images to display on the frontend. The site I was displaying this on is now defunct. All I was using the site for was to check the weather at work in the mornings. My confidence in building with microservices and embedded systems has grown. In our day-to-day coding we might not realize how lucky we are to have packets move smoothly across the network, where being cognizant of how a network might not support the standard methods of transmitting data is beneficial for future troubleshooting and design. Solving problems and knowing the steps to find the right solution, even when that solution feels so wrong is what this exercise was all about!

Leave a Reply

Your email address will not be published. Required fields are marked *