Category: Linux

Building a Small Development Environment with Git, GitLab, Jenkins, and MSBuild

Building a Small Development Environment with Git, GitLab, Jenkins, and MSBuild

It’s been a little while since I posted about a tech project – I’ve spent the first half of the year exploring the Dark Souls universe and watching a bunch of Anime. But now I have a project to do and while researching how to do that project, I found that there doesn’t seem to be a resource on the internet for it.

The Project

My wife is starting a non-profit that is going to need a software platform, and I volunteered to help. However, it’s been a long time since I started something from scratch without any infrastructure, and as I searched the internet, as usual, I found lots of parts of a solution, but not an end-to-end guide to getting a workable build environment built. So I’ll share my notes here in case they happen to be useful to anyone.

My Skills and the Core Tech I’ll Use

I have worked the majority of my career in the Microsoft world, and the project will need a database, so I’m going to center this around a MS SQL Server database, with a C# middle layer/BFF using Entity Framework, and Angular as a front end.

The goal is to have a Development, Test, and Production build system that effectively means that no human interaction is needed with the servers.

I’m going to host this in Azure in a mix of VMs, and I’m going to try to balance security with expediency, with a first goal of getting the whole system working, and the second of gradually hardening the solution. The main issue here is that large scale corporate programming does have a lot more resources, so I want a low-cost reliable system that I can store in the cloud but won’t be a huge burden to manage.

The Software

For this task, I’m going the use the following software.

Git: It’s free, ubiquitous, and a platform I need more experience in. I have been in the TFS (now Azure DevOps Server) world for years, and so I need to get my arms around Git.

GitLab: I wanted a UI for Git for a few reasons. Partly due to my unfamiliarity, and partly because if gives a lot of features out of the box. It also has CI/CD and a ton of other features.

Jenkins: I’m not, actually, going to use the CI/CD pipeline from GitLab, though. I’m going to use Jenkins. I know it better, and it’s great software.

Visual Studio Community Edition: Yes, I could use VS Code, but again, I know VS really well, especially its source control integration.

The Architecture

Basic Lab Setup
Basic Lab Setup

I have an MSDN account, so I’ll use my Azure credits there to flesh this out. I’m going to use 4 VMS to start with:

  • Git server: An Ubuntu server running Git and GitLab. It will have 2 vcpus and 8GB ram.
  • Jenkins Server: a Windows 10 VM running Jenkins. It will have 2 vCPUs and 8GB RAM. It’s on windows so I can run MSbuild and MSDeploy. These might work on Linux, but I see no reason to find out right now.
  • Development server: A windows 10 server with Visual Studio and SQL Server Express. I’ll ultimately move the SQL server to a real box, but SQL instances are expensive, and for now I can live with SQL express. It has  4vCPUs and 32GB RAM.
  • IIS Server: I may or may not end up using IIS (maybe I’ll do an asp.net self-hosted app), but for now, IIS is a pretty good solution.

I’m not doing AD or any sort of domain name service, I’ll be just editing hosts files for this and using internal IPs. For this scale, it’s a bit of a pain to set up, but internal DNS isn’t worth the hassle.

Security to start will be pretty weak, but the goal is to get the environment built and working.

Other Software

In addition to the above, here are other bits of software you’ll need:

  • Host File Editor. It’s for editing host files. It’s one of those great pieces of software you didn’t think you needed until you found it.
  • Google Chrome: I may be using windows, but I have no reason to use Edge.

Next Steps

In the next few posts, I’ll go through the construction and configuration of this environment. The rough steps will be:

  1. Install and configure Git, Gitlab, and a dev machine.
  2. Install and configure Jenkins.
  3. Build the software and the deployer jobs.

What I’m listening to as I do this: DragonForce’s new album ‘Extreme Power Metal‘. DragonForce does one thing, and it does it well. It goes fast. Also, they covered ‘My Heart Will Go On’ on this album. This version of that song should have been the first dance at my wedding (not a household consensus, though).

 

 

 

 

Adding A Custom GeoIP Field to Filebeat And ElasticSearch

As part of my project to create a Kibana dashboard to visualize my external threats, I decided I wanted a map view of where the IP addresses were coming from with geoip data. By default, Filebeat installs several dashboards that I used as inspiration, and saw what could be done, so I set out to imitate them.

Continue reading “Adding A Custom GeoIP Field to Filebeat And ElasticSearch”

Your Home Network Is Under Attack 5000 Times A Day.

We’re always told that the internet is a very dangerous place, but that is a very abstract threat. I have often found myself in a position where I tell people that the internet is dangerous, but I have no real data to back this up.

Therefore, my latest project was a deep dive into trying to figure out, for myself, how likely I was to get attacked if I had a server open to the internet. I know that it’s not a matter of ‘if’, but of ‘when’, but I wanted to quantify the ‘when’ as well as the ‘what’.

The Plan

My plan has 3 phases. Generally, I am going to use an SSH (Secure Shell) server to act as a honeypot to lure in attackers. SSH is the black terminal window that people associate with Linux (and Jurassic Park).

The Bash Shell

SSH uses a server that listens usually on port 22, and when you log into a Linux server, you’re often using that.

So my plan is:

  1. Determine the general external threat: what is my router blocking?
  2. Determine the time it takes for a slightly hidden server that has an ssh port forwarded to it to come under direct attack
  3. Profile the attack on ssh to see how the attack takes place.

Continue reading “Your Home Network Is Under Attack 5000 Times A Day.”

Installing ELK without Docker on Ubuntu 18.04 with LetsEncrypt

In my last post, I set up ELK in a docker container to see if it would meet my needs, but I found that unless I wanted to go very deep with docker, I’d need to do a raw installation of ELK. The main motivation was that I wanted to install an SSL certificate in Kibana using LetsEncrypt from my pfSense box, and building a job that builds docker every 90 days seemed brittle.

One thing I realized was that you do do a lot without installing the ‘L’ in ELK. LogStash and ElasticSearch both provide means to ingest logs. When you install filebeat on your client, you can opt to output to LogStash or to ElasticSearch. I went direct to ElasticSearch for now, though I will likely revisit that later on. Here are two resources that discuss that here and here.

Continue reading “Installing ELK without Docker on Ubuntu 18.04 with LetsEncrypt”

Next Project – Video Surveillance with Unifi Video

Now that I’m done with my WiFi project, and everyone is surfing happily (it seems), I’m ready for my next project. For years now I’ve had a basic video surveillance setup using old Panasonic Security Cameras (BL-VP104W) and one ultra sketchy Chinese camera. You know, one of those with extra backdoor admin accounts named 888888 that you can’t disable. I bought extras that I can’t sell on eBay at any price.

Seriously, dont buy these.
Seriously, don’t buy these.

So I want to upgrade to a better video solution. Fortunately, Ubiquiti offers a solution that is as cost-effective as its infrastructure.

Continue reading “Next Project – Video Surveillance with Unifi Video”

Logging into my Wifi With RADIUS

I have been using my UniFi system for a few months and I’m very pleased. The WiFi is very fast, I can roam around the house seamlessly, and the handoffs are seamless. Now I want to move to the two final stages: advanced access control and then content control for the kids. For access control, I’m going to use RADIUS, specifically pfsense’s freeRADIUS package. I’ve read that this it is a bit arcane, but my experimentation has shown that it’s actually pretty straightforward.

Continue reading “Logging into my Wifi With RADIUS”

Switching my UniFi Controller to Ubuntu and HDMI Problems

I have been troubleshooting a really peculiar problem with my Unifi Setup that perplexed me for several days. Since I use HDMI over IP to stream my cable box to several TVs around the house over cat5, I wanted to isolate the video on its own VLAN. I use Mirabox extenders, and as I was working on my article about putting my video on its own VLAN, I found out that UniFi has a big problem with that. Continue reading “Switching my UniFi Controller to Ubuntu and HDMI Problems”

Installing a LetsEncrypt SSL Certificate with pfSense on an Internal Server

Ever since Google announced that Chrome would mark non-https connections as ‘Not Secure’ I’ve begun to fret about ssl certificates. These serve two purposes. First, they encrypt your data and prevent Man-in-the-middle attacks, and secondly, they verify that the site you visit is the site it claims to be. I used to think that the former was more important, but now I am more of the opinion that identity verification is most important, now that phishing attacks are commonplace. LetsEncrypt has recently stepped in to help solve this problem.

With this in mind, when I saw that my UniFi controller was marked:

The 'Note Secure' view on Chrome.
Not Secure!

I wanted to fix it. Continue reading “Installing a LetsEncrypt SSL Certificate with pfSense on an Internal Server”

Installing The Ubiquiti UniFi Controller On My Centos Server

We’re almost at the fun part of my project, but first I’m going to take another detour to show how to install the Ubiquiti UniFi Controller software onto my main Centos file/utility server.

Update 10/14/18: Don’t do this. Bad idea. I do over why in a later post.

I’m going to explain this in a bit of detail. I remember when I started out with Linux (Solaris, actually) and it was so obscure that I found it frustrating. So I’ll try and explain each step. Continue reading “Installing The Ubiquiti UniFi Controller On My Centos Server”