Windows Command Line and Copy-Paste

Using Copy-Paste in the Windows Command Line isn’t straightforward by default. To paste anything, you have to right-click to open up the context dialog and click insert (or “Einfügen” in German J):

However, the Command Line does have some semi-hidden settings which can simplify things. You have to right-click the cmd.exe window title and enter Properties (“Eigenschaften” in my case):

In the Options tab, you should tick the Quick-Edit-Mode checkbox and press Ok. You can of course adjust the other options to your liking:

From now on, whenever you’ve got something in the copy buffer, a simple right-click pastes the content into the Command Line Window. Neato.

Visual Studio C# Block Outlining

The C# editor in Visual Studio supports collapsing/expanding namespaces, classes and methods (aka the outlining feature).  Unlike the C++ editor, it doesn’t allow to collapse blocks (everything with braces, such as if/else, using, switch, etc.). However, you can retrofit this feature with the C# Outline 2013 extension.

Installation

Navigate to Tools -> Extensions and Updates -> Ctrl + E -> Type “outline” -> Install extension and restart.

Result

After restarting, blocks are outlined and you can collapse / expand them:

Outlined blocks!

It also adds some new shortcuts:

New shortcuts

 

Git with Synology DiskStation

When developing private software projects, you not always want to push it to Github right away if you’ve got the hopes to make some bits of money out of it, right? But just keeping your Git repository on your local disk isn’t the best choice either. If you happen to have a server at home, you should use that to backup your repository. Of course you should do that the Git way, i.e., set up a bare repository and use that as your origin remote to push to.

That’s particularly easy if you own a Synology DiskStation. Not too long ago, an official Git Server installation package was made available:

DSM "Git Server" package description

Enable SSH and user grant access rights

After installation, you get access to a simple GUI where you can allow local users to access Git over SSH:

Git config

Tick the checkbox for your user and press Apply. To make this work, you also need to enable SSH access to your DiskStation. For that, head into the Control Panel, navigate to Terminal & SNMP and tick the Enable SSH Service checkbox. For setting up a bare Git repository, you have to log in over SSH (assuming diskstation is a proper hostname):

$ ssh root@diskstation

Windows users can use PuTTY.

Remote Git repository setup on the DiskStation

This is how to set up my_repo for user sascha on /volume1 (Diskstation default, don’t worry):

$ mkdir -p /volume1/git/my_repo
$ chown -R sascha. /volume1/git/my_repo
$ cd /volume1/git/my_repo
$ git init --bare

Then you can log out from your DiskStation again. On your local machine, you can either directly clone this newly created repository:

$ git clone ssh://sascha@diskstation:/volume1/git/my_repo

Or add it as a remote to your already existing git repository and push your master branch to it:

$ git remote add origin ssh://sascha@diskstation:/volume1/git/my_repo
$ git push --set-upstream origin master

That’s it.

BitTorrent Sync on openSUSE

Recently, I discovered BitTorrent Sync, which seems to satisfy most of my file syncing demands. It’s encrypted client-side, cross-platform and works behind NATs and firewalls. While it is currently still proprietary (who cares, really), it is available for many devices. Besides the usual Windows / Mac binaries, you can find it on Android’s Play Store. Most interestingly, they provide ARM binaries. If you are a happy Synology NAS user, you can add the SynoCommunity package repository directly. That’s been the elevator pitch, check the community forums for more details.

Installation

First of all, you need to add Packman’s Essentials repository to install the btsync package. This is necessary due to the licensing terms of BitTorrent Sync, which don’t allow redistribution. Thus the btsync package will run a script (during installation) that downloads the btsync binary from BitTorrent servers (very much like the flash plugin installer on openSUSE). Either way, you’ll end up with btsync on your disk. For openSUSE Factory, the steps are (as root):

$ zypper ar http://packman.inode.at/suse/Factory/Essentials
$ zypper refresh
$ zypper install btsync

On recent openSUSE releases, systemd allows to start daemons as non-root users. Running btsync under your user rather than root avoids messing up file ownership and allows several people on the same machine to have their own distinct btsync configuration. So for the user saschpe (replace with your username), the commands are (as root):

$ systemctl start btsync@saschpe
$ systemctl enable btsync@saschpe

Configuration

To allow parallel usage for multiple users, the btsync daemon listens on port 8888 + $YOUR_USER_ID per default. So if your user’s UID (check /etc/passwd) is 1000, you can find btsync’s web interface at http://localhost:9888. This is how looks like:

btsync-web

The credentials can be found at $HOME/.btsync/sync.conf. The interesting part is the auto-generated password (which you could change), the username will match your Unix user account. However, you may also want to change the listen address to something different. So this is the config section you want to adjust:


...
"webui" :
{
"listen" : "0.0.0.0:9888",
"login" : "saschpe",
"password" : "supersecretpasswordhere"
}
...

Usage

Sharing folders and further setup can be done directly from the web interface, no need to mess with the config file again. Have fun.

eBay’s funny mail restrictions vs. the plus extension

Since eBay recently lost a complete database dumb, I thought let’s join the other 145 million guys and change me good old password. While doing that I wanted to update my mail address too. BTW, I recently changed to mailbox.org and I am very (very very) happy with it. Unfortunatley though eBay’s very (very very) secure e-mail address validation disallows having the eBay username as localpart (as in localpart@domain.com). Wow I thought, that is what I call secure by default! Funny side note, my old e-mail address also includes my eBay user name, but nobody complained about that some 10 years ago. Well then I thougth, let’s check support. After almost going nuts in their craptastic present-stupid-solutions-but-avoid-revealing-the-bloody-support-form manor, I went for the (free) hotline. So after a 28 minutes long patience test (which I obviously won), I was told by a very friendly lady that she isn’t allowed to change the mail address by hand. German data security laws, she said. Fuck that, I thought :-) She also recommended not using this mail address ;-) But that led me to suddenly remember that mailbox.org supports the plus-extensions (or whatever it’s called in the RFCs). So I ended up trying username+ebay@mailbox.org and it worked! So kudos to eBay for not disallowing plus signs in mail addresses (no sarcasm here, many services do). Even better, the next time they loose all their users data, spammers will only get the alias address and I can just block that. Nice!

Dynamic iptables port-forwarding for NAT-ed libvirt networks

Libvirt is particularly awesome when it comes to managing virtual machines, their underlying storage and networks. However, if you happen to use NAT-ed networking and want to allow external access to services offered by your VMs, you’ve got to do some manual work. The simplest way to get access is to set up some  iptables rules to do port-forwarding. So for quite a while, I had the following in my /etc/init.d/boot.local and things just worked:

HOST_IP=10.120.4.195

function iptables_forward() {
    read HOST_PORT FORWARD_IP FORWARD_PORT BRIDGE <<< $2
    # Drop rules just in case they are already present:
    iptables -t nat -D PREROUTING -p tcp -d $HOST_IP --dport $HOST_PORT -j DNAT --to $FORWARD_IP:$FORWARD_PORT 2> /dev/null
    iptables -D FORWARD -o $BRIDGE -p tcp --dport $FORWARD_PORT -j ACCEPT 2> /dev/null
    echo "iptables_forward: Forward service $1 from $HOST_IP:$HOST_PORT to $FORWARD_IP:$FORWARD_PORT"
    #iptables -t nat -A PREROUTING -p tcp -d $HOST_IP --dport $HOST_PORT -j DNAT --to $FORWARD_IP:$FORWARD_PORT
    #iptables -I FORWARD -o $BRIDGE -p tcp --dport $FORWARD_PORT -j ACCEPT
}

declare -A service
# Declare array of forwarding rules for VM services in the following form:
#service["VM_SERVICE_NAME"]="HOST_PORT FORWARD_IP FORWARD_PORT"

# libvirt 'default' network:
#service["devstack_dashboard"]="1011 192.168.122.100 80 virbr0"
#service["obs_api"]="1011 192.168.122.80 4040 virbr0"
#service["obs_webui"]="1022 192.168.122.80 80 virbr0"
#service["quickstart_crowbar"]="1030 192.168.122.101 3000 virbr0"
#service["quickstart_dashboard"]="1031 192.168.122.101 443 virbr0"
#service["quickstart_chef_webui"]="1032 192.168.122.101 4040 virbr0"
#service["quickstart_ssh"]="1033 192.168.122.101 22 virbr0"

# libvirt 'cloud' network:
service["cloud_crowbar"]="1100 192.168.124.10 3000 virbr1"
service["cloud_dashboard"]="1101 192.168.126.2 80 virbr1"
service["cloud_dashboard_ssl"]="1102 192.168.126.2 443 virbr1"

for key in ${!service[@]} ; do
    iptables_forward "$key" "${service[$key]}"
done

Pretty simple. However, with systemd things got more complicated. Not only is /etc/init.d/boot.local not evaluated anymore, but it likes to fight with libvirt over when to create bridges (and VLANs). Thus I had to manually invoke the script after libvirt was running.  After re-reading libvirt’s awesome documentation, it was clear that this really rather belongs into a hook script. For qemu domains, the script has to be put in /etc/libvirt/hooks and named qemu. I has to comply to this rather simply interface:

/etc/libvirt/hooks/qemu VIR_DOMAIN ACTION ...

Where VIR_DOMAIN is the exact name of the libvirt domain (virtual machine) for which you want to add / remove iptables port-forwarding rules. ACTION is either “start”, “stopped” or “reconnect”. It could be something like this Python script:

#!/usr/bin/python

"""Libvirt port-forwarding hook.

Libvirt hook for setting up iptables port-forwarding rules when using NAT-ed
networking.
"""
__author__ = "Sascha Peilicke <saschpe@gmx.de>"
__version__ = "0.1.1"


import os
import json
import subprocess
import sys


CONFIG_PATH = os.path.dirname(os.path.abspath(__file__))
CONFIG_FILENAME = os.path.join(CONFIG_PATH, "qemu.json")
CONFIG_SCHEMA_FILENAME = os.path.join(CONFIG_PATH, "qemu.schema.json")
IPTABLES_BINARY = subprocess.check_output(["which", "iptables"]).strip()


def host_ip():
    """Returns the default route interface IP (if any).

    In other words, the public IP used to access the virtualization host. It
    is used as default public IP for guest forwarding rules should they not
    specify a different public IP to forward from.
    """
    if not hasattr(host_ip, "_host_ip"):
        cmd = "ip route | grep default | cut -d' ' -f5"
        default_route_interface = subprocess.check_output(cmd, shell=True).decode().strip()
        cmd = "ip addr show {0} | grep -E 'inet .*{0}' | cut -d' ' -f6 | cut -d'/' -f1".format(default_route_interface)
        host_ip._host_ip = subprocess.check_output(cmd, shell=True).decode().strip()
    return host_ip._host_ip


def config(validate=True):
    """Returns the hook configuration.

    Assumes that the file /etc/libvirt/hooks/qemu.json exists and contains
    JSON-formatted configuration data. Optionally tries to validate the
    configuration if the 'jsonschema' module is available.

    Args:
        validate: Use JSON schema validation
    """
    if not hasattr(config, "_conf"):
        with open(CONFIG_FILENAME, "r") as f:
            config._conf = json.load(f)
        if validate:
            # Try schema validation but avoid hard 'jsonschema' requirement:
            try:
                import jsonschema
                with open(CONFIG_SCHEMA_FILENAME, "r") as f:
                    config._schema = json.load(f)
                jsonschema.validate(config._conf,
                                    config._schema,
                                    format_checker=jsonschema.FormatChecker())
            except ImportError:
                pass
    return config._conf


def iptables_forward(action, domain):
    """Set iptables port-forwarding rules based on domain configuration.

    Args:
        action: iptables rule actions (one of '-I', '-A' or '-D')
        domain: Libvirt domain configuration
    """
    public_ip = domain.get("public_ip", host_ip())
    # Iterate over protocols (tcp, udp, icmp, ...)
    for protocol in domain["port_map"]:
        # Iterate over all public/private port pairs for the protocol
        for public_port, private_port in domain["port_map"].get(protocol):
            args = [IPTABLES_BINARY,
                    "-t", "nat", action, "PREROUTING",
                    "-p", protocol,
                    "-d", public_ip, "--dport", str(public_port),
                    "-j", "DNAT", "--to", "{0}:{1}".format(domain["private_ip"], str(private_port))]
            subprocess.call(args)

            args = [IPTABLES_BINARY,
                    "-t", "filter", action, "FORWARD",
                    "-p", protocol,
                    "--dport", str(private_port),
                    "-j", "ACCEPT"]
            if "interface" in domain:
                args += ["-o", domain["interface"]]
            subprocess.call(args)


if __name__ == "__main__": 
    vir_domain, action = sys.argv[1:3]
    domain = config().get(vir_domain)
    if domain is None:
        sys.exit(0)
    if action in ["stopped", "reconnect"]:
        iptables_forward("-D", domain)
    if action in ["start", "reconnect"]:
        iptables_forward("-I", domain)

It has a very simple configuration file that is expected to live at /etc/libvirt/hooks/qemu.json:

{
    "cloud-admin": {
        "interface": "virbr1",
        "private_ip": "192.168.124.10",
        "port_map": { 
            "tcp": [[1100, 3000]],
            "udp": [[1200, 163]]
        }
    },
    "cloud-node1": {
        "interface": "virbr1",
        "private_ip": "192.168.126.2",
        "port_map": {
            "tcp": [[1101, 80],
                    [1102, 443]]
        }
    }
}

With that in place, iptables rules are added and removed when the domain is started / stopped. Pretty neat, huh? You can find the full code together with some tests and documentation on the libvirt-hook-qemu Github repository.