Streams, Promises and Throttles

Recently in my journey I came across a nice little challenge to read a really large csv, pull data from it and then call a couple of APIs IOT to get some data then output a CSV after processing was complete. The file I was provided basically contained a long list if IDs which I needed to use to get information from one API then use that information to call Watson. I hadn’t flexed my muscles with the stream API for a while so it was fun. I just wanted to share this example in case other are looking for a way to solve a similar problem.

Minecraft on Ubuntu – Forge Upgrade

To upgrade your Minecraft Server on Ubuntu to a more recent version of forge:

Download the installer and universal package for the version of forge you want.
Transfer them to the server.
Run the installer: `java –jar forge-X.X.X-X.X.X.XXX–installer.jar —installServer`
Restart the server: `java -Xms1024M -Xmx8192M -jar ~/minecraft/forge-1.12.2-14.23.3.2675-universal.jar noguiadsls`

Hyperledger Fabric v1.0 on a Raspberry Pi Docker Swarm – Part 4

And now for the thrilling conclusion to… HLv1_RPiDS! (<— a what now?)

In this section we’ll go over the steps I take to launch the network and talk through some of the configuration sections to watch out for as you setup your own.

But first a quick proof of work demonstration:

Hyperledger Fabric v1.0 on a Raspberry Pi Docker Swarm – Part 3

As I said in my last post this step is fairly easy and overall gave me the least amount of trouble.

Before we dive in I want to mention some prerequisite equipment you’ll need. You should have at least 3 Raspberry Pis (<– this looks so wrong to me… Pies?) to setup up a true Raspberry Pi Docker Swarm. This means you’ll need SD card for each and a way to power them, I recommend a USB power hub like this (something with 60 watts and 6 ports).

Hyperledger Fabric v1.0 on a Raspberry Pi Docker Swarm – Part 2

I want to thank all of the people who commented on my first post about this project, I can’t believe how much time has passed since I first wrote about this. I actually did get it running but ran out of time to document it all while getting it working… I planned show it at a conference for work and ran into significant hurdles along the way so my documentation slipped a bit. I’m going to finish this out thanks to the extra motivation I received from everyones interest. So let’s get to it!

Mastering the build step is a pretty decent challenge. In this article I’ll pass along some tips and hints on how I pulled it off.

Hyperledger Fabric v1.0 on a Raspberry Pi Docker Swarm – Part 1

There have already been articles published on the use cases for using IoT with a private Blockchain. The possibilities are really exciting but what if we could run the blockchain ON our IoT network. That sounds like a fun project to me!

With that goal in mind and a bit of research I was led to Hyperledger Fabric. To learn more about it check this out: Hyperledger Overview. Getting Hyperledger Fabric to run on Raspberry Pi presented several major hurdles:

No one had compiled the project for the ARM architecture before.
There wasn’t any documentation on how to run the Hyperledger on a Docker Swarm.

But hurdles are what make projects fun right? So lets go through the steps so that you can setup your own.

Lets get started!

Putting a newer version of Node.js on LinkIt Smart 7688 Duo

Today I’m going to step you through putting a newer version of Node JS on your LinkIt Smart 7688. The default version of node available is 0.12.7 which, let’s face it, is completely outdated and essentially useless for to any serious Node.js developers. Now there are some ongoing challenges with putting a completely up to date version of Node on the MIPS architecture. This is because there are key libraries needed to build Node JS that are not updated on MIPS yet… so since those libraries out out of date we can only get so far.

Setup your linux machine.

I use Ubuntu server 16.04.2 LTS at the time of writing this tutorial. Get Virtual Box, download an Ubunut 16 .ISO, create a new VM, make sure your disk is around 50GB (5GB is not enough… found this out the hard way), select the ISO you downloaded, install the operating system (be sure to add SSH for convenience).

I alway use SSH for copy paste commands. To setup you VM so you can ssh to it:

Open the setting for the VM (you can do this while its running)
Go to Network > Adapter 1 > Advanced > Port Forwarding
Forward 127.0.0.1:2200 to 10.0.2.15:22
- Check your ifconfig to verify your VMs IP address

SSH to it from your host with this:

ssh usename@127.0.0.1 -p 2200

Alternatively you can put the adapter in Bridge mode… then check ifconfig for the VMs IP.

Setup for the build

Following for guide here with modified instruction for things they forgot:

https://docs.labs.mediatek.com/resource/linkit-smart-7688/en/tutorials/firmware-and-bootloader/build-the-firmware-from-source-codes

and with help from the nxhack who is running this repo:

https://github.com/nxhack/openwrt-node-packages/tree/for-15.05

Add the python package (if you didn’t install during Ubuntu OS install)
```
sudo add-apt-repository ppa:fkrull/deadsnakes
sudo apt-get update
```

Install prerequisite packages to build the firmware:

sudo apt-get install git g++ libncurses5-dev subversion libssl-dev gawk libxml-parser-perl unzip wget build-essential python2.7

Download the OpenWrt CC source codes:

git clone git://git.openwrt.org/15.05/openwrt.git

Prepare the default configuration file for feeds:
cd openwrt cp feeds.conf.default feeds.conf
Add the LinkIt Smart 7688 development board’s and NodeJS to the builds feed:
```
sudo nano ./feeds.conf
```
Add the following lines to the bottom of the file:
src-git linkit https://github.com/MediaTek-Labs/linkit-smart-7688-feed.git src-git node https://github.com/nxhack/openwrt-node-packages.git;for-15.05
Update the feed information for all available packages to build the firmware:
./scripts/feeds update

Change the packages installed as default:

wget https://gist.githubusercontent.com/nxhack/7ced4d866a59ebc72737589b49a220f8/raw/1bebfe9f6091f55a5856bc4af00da456a4995b09/mtk-linkit.patch
patch -p1 < mtk-linkit.patch

Fix build error of depend on node.js version:

wget https://gist.githubusercontent.com/nxhack/4357d51918ba8f3cb5cc00080ad0815c/raw/e96758224ade8cb224523aedf5ea3249a6a59425/MRAA.patch
patch -p1 < MRAA.patch

Hack for wifi driver so build completes
Copy kernel objects for support kernel 3.18.45:
see : MediaTek-Labs/linkit-smart-7688-feed#37

cp ./feeds/linkit/mtk-sdk-wifi/wifi_binary/mt_wifi.ko_3.18.44 ./feeds/linkit/mtk-sdk-wifi/wifi_binary/mt_wifi.ko_3.18.45
cp ./feeds/linkit/mtk-sdk-wifi/wifi_binary/mt_wifi.ko_3.18.44_all ./feeds/linkit/mtk-sdk-wifi/wifi_binary/mt_wifi.ko_3.18.45_all

Install all packages:
./scripts/feeds install -a

Use node.js custom packages:

rm ./package/feeds/packages/node
rm ./package/feeds/packages/node-arduino-firmata
rm ./package/feeds/packages/node-cylon
rm ./package/feeds/packages/node-hid
rm ./package/feeds/packages/node-serialport
./scripts/feeds install -a -p node

Prepare the kernel configuration:
make menuconfig
- Select the following options:
  - Target System: Ralink RT288x/RT3xxx
  - Subtarget: MT7688 based boards
  - Target Profile: LinkIt7688
- (Optional) GO into Languages > Node.js > Configuration (under the node package) > Select your desired version.
- (Optional) Enable modules you want (Caution there is a 30MB limit so not all will fit)
  - (Recommended) node-npm (it is a separate module in v6)
- Save and exit (use the default configuration file without any modification)
!!!BEFORE YOU START!!!
RUN THIS FROM YOUR VM NOT OVER SSH
This command will take a while and if your ssh pipe breaks so will your build
Start the compilation process:
```
make
```
There are several options you can use with the make command that are helpful.
– V=99 (gives verbose output during build) or
– V=1 (shows error, warnings, and notes “less verbose”)
– &> output.log ( on the end – stores output in a log for later viewing)

A Few Hours Later…

After the build process is finished successfully, the resulted firmware file will be under “bin/ramips/openwrt-ramips-mt7688-LinkIt7688-squashfs-sysupgrade.bin”. If its’ not there check the output in your log file or on your screen. Depending on the hardware resources of the host environment, the build process may take more than 2 hours.
You can use this file to update the firmware through Web UI or rename it to lks7688.img to update through a USB drive.

A big thanks to nxhack for making this build possible!

Download images for the version of Node you want:

lks7688-node-v4.img

MD5: d7f724da93a1d916bf777f80516a0f33

SHASUM: f74182c70b937909ad1cba6f97e40b5dd3891962

lks7688-node-v6.img

Includes NPM.

MD5: a17c672f87c4b8fa49a253d5534a9229

SHASUM: 8b19b20d23f1faa94c9dd084ca0271904d1dfa5e

Add a New Page to ReactGo

Adding a new page to your reactGo project can be a lengthy process. I’m hoping to write a yeoman generator to do this for me in the near future but for now… Here are the steps.

Steps:

Add a new route to your app >routes.js
Add Something like: <Route path="routeName" component={page} onEnter={requireAuth} fetchData{[fetchItemData, fetchCartData]} />
to the Route section.
Add the necessary supporting imports.
Add a new entry to the navigation.
Create a new page in the pages folder.
Duplicate existing page, then open. Update references to new component name.
Add reference to app > pages > index.js
export { default as ContianerName } from 'pages/ContainerName';
Create a new data fetcher (if necessary)
Duplicate existing and update path to get data from server and action type to dispatch.
Update the app > fetch-data > index.js with the new reference.
export { default as fetchMyData } from './fetchMyData';
If a new data set is needed follow these steps: Add a New Dataset to the Store
Create a new Container
Duplicate an existing component then design and build your page.
After your pages design is complete componentatize what you should.

Questions for yeoman generator:

Provide Route name: String
Provide page/component name: String
Does the page already exist? Boolean
If No => Provide Page Description: String
Would you like to generate a new CSS Module for the page? Boolean
Auth required? Boolean
Does it need to fetch any data? Boolean
- If yes => How many? Number
- For count:
  - Provide Data Fetcher name: String
  - Does it already exist? Boolean
  - If No => Does dataset exist?
    - If No => Start dataset generator
    - If Yes => Provide path to fetch data from server.

Dockerizing ReactGo

First up you’ll need to be able to successfully run a build for your current ReactGo app. I assume you have docker installed.

npm run build

Once you’ve got a successful build on your hands go ahead and create your dockerfile. Example one below:

FROM node:boron

# Create app directory
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

# Install app and dependencies
COPY package.json /usr/src/app/
RUN npm install

# Copy build
COPY compiled /usr/src/app/compiled
COPY public /usr/src/app/public

# Set Environment Variables
ENV MONGODB_URI mongodb://mongo/MyAppCollection
ENV NODE_ENV production
ENV PORT 80

EXPOSE 80

CMD ["node", "compiled/server.js"]

You can use a later version of Node if you like I built and tested my on v6.10.

docker build -t <image_name> .

NOTE: DO NOT FORGET THE PERIOD AT THE END (that caused me a serious headache one day…)

While that is building lets launch a mongo container to run our database (we’ll link to it later). I assume you’ve already pulled mongo from the docker hub: docker pull mongo

docker run -d --name <db_container_name> mongo

Now let test our container locally… I’m going to bind our exposed 80 port to port 80 on localhost to demonstrate how it’s done. This same principal applies when launching a container out in a production docker environment.

docker run -d -p 127.0.0.1:80:80 -it --link <db_container_name>:mongo --name <app_container> <image_name>

NOTE: notice the MONGODB_URI ENV variable we set in the docker file. Take special note of underlined part that follows: the mongodb://mongo/MyAppCollection. This MUST match the link name you give in your run command (e.g. <db_container_name>:mongo).

Check your local host on port 80 and you should see your app. If not check the logs by using the docker logs <container_ID> command. Container IDs can be found with docker ps -a command.

After you’ve successfully tested your container you’re ready to push the image to your registry. You can use docker hub but I’m using Bluemix, since it’s private and I get it for free ;). After you login to your registry tag your image for release:

Bluemixers need your name space? try bx ic namespace-get

docker tag <image_name> registry.ng.bluemix.net/<name_space>/<app_name>:<version_tag>

The version tag is optional but recommended. It will default to latest otherwise. Then push it (push it real good)

docker push registry.ng.bluemix.net/<name_space>/<app_name>:<version_tag>

Then wait… forever…

If your on Bluemix you’ll probably want to push up a mongo container as well. (remember to start one up so we can link to it like we did on local host)

For those of you who are not using Bluemix substitute the bx ic commands below for your registry (should be just docker for most of the world)

bx ic cpi mongo registry.ng.bluemix.net/<name_space>/mongo

From there I usually create a dummy container with the Bluemix GUI (website) so I can provision a public IP through Bluemix which I can then reuse when I launch a container with this command. There is probably a way to provision an IP through CLI but I’m too lazy to look it up.

bx ic run -d -p <my_public_ip>:80:3000 -it --link <db_container_name>:mongo --name <app_container_name> <name_space>/<image_name_or_id>

React, Emmet, ESLint, Babel Packages

First make sure you have the following packages added to your project:

npm install --save-dev babel-eslint eslint eslint-config-airbnb eslint-plugin-react eslint-plugin-jsx-a11y eslint-plugin-import mocha

On a fresh instal of Sublime text install the following with package control.

Babel
Emmet
SublimeLinter
SublimeLinter-contrib-eslint

After installation restart Sublime. Now your files will have linting on them so you can see how nasty your code is. 😉

Sample .eslintrc for refference

{
  "parser": "babel-eslint",
  "extends": "airbnb",
  "env": {
    "browser": true,
    "node": true,
    "mocha": true
  },
 "parserOptions": {
    "ecmaVersion": 6,
    "sourceType": "module",
    "ecmaFeatures": {
      "forOf": true,
      "jsx": true,
      "es6": true,
      "experimentalObjectRestSpread" : true
    }
  },
  "rules": {
    "comma-dangle": 0,
    "indent": 0,
    "react/prop-types": 0,
    "react/jsx-indent-props" : 0,
    "linebreak-style": 0,
    "react/jsx-closing-bracket-location" : 0,
    "object-curly-spacing" : 0,
    "arrow-body-style": 0,
    "no-console": 0,
    "max-len": 0,
    "prefer-template": 0,
    "import/no-unresolved": 0,
    "global-require": 0,
    "no-underscore-dangle": 0
  },
  "plugins": [
    "react"
  ],
  "globals": {
    "__DEVSERVER__": true,
    "__DEVCLIENT__": true
  }
}