Software Collections 2.0 now in BETA – new and shiny


softwarecollections-logo-colorfulIt seems like just a few months ago when we introduced Red Hat Software Collections 1.0 (RHSCL), followed by 1.1 and 1.2 will lots of additions and updates.

Today, Red Hat has announced Red Hat Software Collections 2.0 with a truck load of important languages, tools, databases and web servers – including the addition of a new component:  Passenger.  Here’s the list:

  • Python 3.4the latest stable, major release of Python 3 and includes a number of additional utilities and database connectors for MySQL
  • PHP 5.6 – featuring numerous improvements, additions and a streamlined upgrade path for migrating from past versions
  • Perl 5.20 – a recent stable release of Perl shipped with a set of additional utilities, scripts, and database connectors for MySQL and PostgreSQL
  • Ruby 2.2 – and, in its own collection, Rails 4.1 give users the ability to access and install an updated version of Ruby without necessarily having to install an updated version of Rails.
  • MySQL5.6 – inclusive of enhancements to InnoDB for higher transactional throughput, partitioning improvements for querying and managing huge tables, and better performance monitoring
  • MariaDB 10 – a recent stable release of this easy-to-adopt database alternative to MySQL
  • PostgreSQL 9.4 – featuring the new JSONB datatype, increased scalability with Logical Decoding, the foundation for new replication tools such as Bi-Directional Replication, and several additional enhancements that contribute to improved performance
  • MongoDB 2.6 – a high-performance, cross-platform document database features comprehensive core server enhancements, enhanced scalability and index intersection
  • NEW TO RHSCL: Passenger 4.0 – a modern web and application server for Ruby, Passenger 4.0 has been optimized for performance, memory usage and ease-of-use

Red Hat Software Collections 2.0 Beta also includes many updates and enhancements to existing collections, including:

  • Maven 3.0.5 – a recent stable release of the popular build automation tool for Java projects that describes how software is built and all associated dependencies
  • Python 2.7 – now includes python-wheel, python-pip, and all associated dependencies
  • Thermostat 1.2 – adds event-based profiling, an improved sampler profiler, Maven archetypes and visual improvements to the Swing client and charts
  • nginx 1.6 – a recent stable release of nginx, a high performance, open source HTTP sever and reverse proxy option
  • DevAssistant 0.9.3 – a useful tools for setting up development environments, publishing code and other related tasks, this latest, stable version includes several bug fixes and full backwards compatibility
  • Node.js 0.10.33 – previously only available through an unsupported tech preview, the latest stable release of this modern programming platform is now fully supported

All of the above are part of the majority of Red Hat Enterprise Linux subscriptions.

New to Software Collections?  Read about them on the product pages.

See the entire release here.

 


For more information about Red Hat Software Collections or Red Hat Developer Toolset, visit developer.redhat.com/RHEL.


Red Hat Developer Toolset 3.1 now generally available


gnu logoToday, Red Hat has announced the general availability of Red Hat Developer Toolset 3.1. Available through the Red Hat Enterprise Linux Developer Program and related subscriptions, Red Hat Developer Toolset 3.1 streamlines application development on the world’s leading enterprise Linux platform, enabling developers to compile applications once and deploy across multiple versions of Red Hat Enterprise Linux.

Updates include: GCC 4.9.2 (the latest stable upstream version of GCC), Eclipse 4.4.2, GDB 7.8.2, elfutils 0.161, memstomp 0.1.5, SystemTap 2.6, Valgrind 3.10.1, Dyninst 8.2.1, and ltrace 0.7.91.

Learn more about the Developer Toolset – see the product pages.

See the full press release.


For more information about Red Hat Software Collections or Red Hat Developer Toolset, visit developer.redhat.com/RHEL.


Scala vs. Node.js as a RESTful backend server


Scala

VS.

node-jsI’ve been involved with full-stack development for a while now, especially stacks involving single page apps. When choosing to go with a single page webapp the backend concerns change. While any backend will do the job (think ruby, python, java, etc.) more emphasis is placed on the front-end stack as most of the time is spent in Javascript and less in the backend language since that is not where the UI logic resides. This is liberating in some senses as it allows one more freedom as language deficiencies become less of a problem. You don’t have to decide between a language based on how you like the backend html templating or framework, since that is no longer involved.  Think python tornado templates, JSF, or jade, three drastically different templating choices.  But none of that matters when going with a single page app in the browser. (more…)


For more information about Red Hat Software Collections or Red Hat Developer Toolset, visit developer.redhat.com/RHEL.


Introducing the Atomic command


RH_Icon_Container_with_App_Flat

Introducing the Atomic command

‘/usr/bin/atomic’

The atomic command defines the entrypoint for Project Atomic hosts.  On an
Atomic Host, there are at least two distinct software delivery vehicles; Docker (often used in combination with the traditional RPM/yum/dnf), and rpm-ostree to provide atomic upgrades of the host system.

The goal of the atomic command is to provide a high level, coherent entrypoint to the system, and fill in gaps in Linux container implementations.

For Docker, atomic can make it easier to interact with special kinds of containers, such as super-privileged debugging tools and the like.

The atomic host subcommand wraps rpm-ostree, click to read more …

The atomic command is also available on non Atomic Platforms.  You can run it on a standard RHEL, Fedora and Centos OS.  We would like to see it get used on other platforms in the future.

Container Image as a software delivery mechanism

The most exciting things about Docker:

Docker images are a new way of delivering software applications to a host operating system.

But, in a lot of ways Docker falls short as a software delivery mechanism.

The atomic command is our effort to close the gaps.

Red Hat has been using RPM to ship software for over 18 years now.  We can look at some of its features to see the short comings.

One of the big features of RPM that Docker image is missing is the mechanism to launch the application at boot time.  When I install the httpd package using yum/rpm it includes the systemd unit file used to launch it at boot time.

We would like to be able to ship one object that a user could easily install.  Currently when you want to install software using Docker images there are two different objects,  You do a Docker pull to install the image, then you build a systemd unit file to run the container, or you execute a complex Docker or Kubernetes command to have the software run as a Docker service.  Bottom line is the developer needs to not only create the Docker image, but he also needs to publish recommended installation procedures. The installation procedures would be easy to misconfigure and we could end up with a support headache.

Software Installation

We need a way for a container to ‘install’ itself in the system to be automatically restarted.

I introduced the concept of something I called a Super Privileged Container back in November.

http://developerblog.redhat.com/2014/11/06/introducing-a-super-privileged-container-concept/

The idea here is to allow a container the ability to see and potentially to manipulate the container hosts. One large use of this would be to allow a container to install itself.

Problem Statement

I am developing a new Apache PHP based application, that I would like to allow my customers to install.  I want to use a systemd unit file.  And I would like to allow my customers to install multiple copies of my application that they could customize to run for different accounts.

The current method for users would be to write up a complicated installation procedure where each user could cut and paste a systemd unit file, and copy it into /etc/sytemd/system/APP.service.  They could download content and setup volume mounts for log directories, data directories and config directories. Perhaps even setup some default data.

The LABEL Patch

We worked with upstream Docker community for over a year to get the LABEL patch, into Docker, which finally got merged into Docker-1.6 package.  This patch allows a software developer to add additional JSON Fields (LABEL) into the Docker Image.  For example I could add a LABEL field to my Docker file like the following.

LABEL INSTALL="docker run --rm --privileged -v /:/host -e HOST=/host -e LOGDIR=${LOGDIR} -e CONFDIR=${CONFDIR} -e DATADIR=${DATADIR} --name NAME -e NAME=NAME -e IMAGE=IMAGE IMAGE /bin/install.sh"

Note that this docker run command is actually a Super Privileged Container command, in that it is privileged and the host OS is mounted at /host within the container.

If an application developer added an LABEL INSTALL line like that in a container image named apache_php, users could examine the docker image for the install line and execute the command.  One problem is the command has NAME and IMAGE embedded in it rather then the container name and the image name.

Better yet the user could let the atomic command do install the application for him.

atomic install apache_php

The atomic command will:

  • The atomic command pulls the apache_php image from a registry if it is not currently installed.
  • The atomic command reads the LABEL INSTALL line from the container image json file.
  • The atomic command replaces any IMAGE values that it sees with the image name installed.  This means the -e IMAGE=IMAGE IMAGE will get substituted with -e IMAGE=apache_php apache_php.
  • The atomic command also allows you to specify a name for your container, and defaults to the image name if the container name is not specified.  In this case since the user did not specify the container name, atomic command will replace NAME with apache_php.  (–name apache_php -e NAME=apache_php).
  • The atomic command generates three directory names and sets passes them in as environment variables ‘LOGDIR=/var/log/NAME, DATADIR=/var/lib/NAME and CONFDIR=/etc/NAME’, where NAME is substituted with the container name (apache_php).  These directories can be used by the container installation procedure to create initial content and eventually could be volume mounted from the host.

If a user wanted to install multiple copies he could just execute

atomic install -n customer1 apache_php
atomic install -n customer2 apache_php

And two containers would be installed and ready to run.

Notice the LABEL INSTALL line in my example executes the /bin/install.sh script that we packaged into the container.  This allows the developer to imbed his installation script into the container image.  Since we are running the container as a SPC we are volume mounting / at /host within the container.

We would like /host to become the standard location for mounting / into a container.  We set the environment variable $HOST within the container to point at /host.  This allows the application developer to write scripts that install content ralative to $HOST/PATH. For example the install.sh might be creating a systemd unit file on the host in /etc/systemd/system/APP.service.  If the install.sh creates the file in ${HOST}/etc/sytemd/system/APP.service, the script would work in an SPC container where -e HOST=/host or if he ran it directly on his test machine outside a container and $HOST would not be set.

The install.sh could also use the $CONFDIR, $LOGDIR and $DATADIR to setup additional content for the container.

Here is my example

/bin/install.sh

#!/bin/sh
# Make Data Dirs
mkdir -p ${HOST}/${CONFDIR} ${HOST}/${LOGDIR}/httpd ${HOST}/${DATADIR}

# Copy Config
cp -pR /etc/httpd ${HOST}/${CONFDIR}

# Create Container
chroot ${HOST} /usr/bin/docker create -v /var/log/${NAME}/httpd:/var/log/httpd:Z -v /var/lib/${NAME}:/var/lib/httpd:Z --name ${NAME} ${IMAGE}

# Install systemd unit file for running container
sed -e "s/NAME/${NAME}/g" etc/systemd/system/httpd_template.service > ${HOST}/etc/systemd/system/httpd_${NAME}.service

# Enabled systemd unit file
chroot ${HOST} /usr/bin/systemctl enable /etc/systemd/system/httpd_${NAME}.service

Notice how the install script is creating directories on the host for the container.  Also it modifies the systemd httpd template file below into a systemd unit file and enables the service.

/etc/systemd/system/httpd_template.service

[Unit]
Description=The Apache HTTP Server for NAME
After=Docker.service

[Service]
ExecStart=/usr/bin/docker start NAME
ExecStop=/usr/bin/docker stop NAME
ExecReload=/usr/bin/docker exec -t NAME /usr/sbin/httpd $OPTIONS -k graceful

[Install]
WantedBy=multi-user.target

When the installation is done the service is ready to run.  And will run on reboot.

Software Removal

The atomic command can also be used to uninstall software.  It will use the LABEL UNINSTALL option if available.

In our example we will use a LABEL like:

LABEL UNINSTALL="docker run --rm --privileged -v /:/host -e HOST=/host -e IMAGE=IMAGE -e NAME=NAME IMAGE /bin/uninstall.sh

Then the user can execute the following uninstall command:

atomic uninstall apache_php

The atomic command will execute the /bin/uninstall.sh script

/bin/uninstall.sh

#!/bin/sh
chroot ${HOST} /usr/bin/systemctl disable /etc/systemd/system/httpd_${NAME}.service
rm -f ${HOST}/etc/systemd/system/httpd_${NAME}.service

Notice the script disables the service and then remove the unit file.

Finally the atomic uninstall will attempt to docker rm a container name if the name is specified or default to the image name.

If the container name is the same as the image name atomic uninstall with also docker rmi the container image.

How do I run the application?

Problem Statement

  • My application is nicely rolled into a container images.
  • My application run mostly confined but needs additional privileges?
    — How do I tell the user to run it?</b></p>

Lets look at an example

The FreeIPA team has been experimenting with running the IPA daemons as separate containers.  One daemon they currently use is ntpd.  The nptd container needs to run with --cap_add SYS_TIME, in order to adjust the system hosts time.  The ntpd container developer has to tell users to run the container with the following command.

Docker run -d -n ntpd --cap_add SYS_TIME ntpd

The atomic command supports another label LABEL RUN, which the application developer can use to define how his application can be run.

FROM rhel7
RUN yum -y install ntpd; yum -y clean all
LABEL RUN=&quot;docker run -d -n NAME --cap_add SYS_TIME IMAGE&quot;
CMD /usr/bin/ntpd

Now if the user examined the Docker image, he would know exactly how to run the container. He could inspect the installed image and then cut and paste the image line.   We automate this process by adding the atomic run command.  A user only needs to execute the following command:

atomic run ntpd

This will do a Docker pull of the ntpd container package onto your host and then execute the label RUN command, if it exists.  If the command does not exist the command will default to a

Docker create -ti -n ntpd ntpd

  • This gives us the ability to define how a specific container expects to be run. Specifically this includes the privilege level required, as well as special mounts and host access, etc.

Other features of the atomic command.

One of the key features of rpm is to list information about a package.

# rpm -qi Docker
Name        : Docker
Version     : 1.5.0
Release     : 25.git5ebfacd.fc23
Architecture: x86_64
Install Date: Thu 26 Mar 2015 03:05:47 PM EDT
Group       : Unspecified
Size        : 21735169
License     : ASL 2.0
Signature   : (none)
Source RPM  : Docker-1.5.0-25.git5ebfacd.fc23.src.rpm
Build Date  : Thu 26 Mar 2015 01:01:50 AM EDT
Build Host  : buildhw-05.phx2.fedoraproject.org
Relocations : (not relocatable)
Packager    : Fedora Project
Vendor      : Fedora Project
URL         : http://www.Docker.com
Summary     : Automates deployment of containerized applications
Description :
Docker is an open-source engine that automates the deployment of any application as a lightweight, portable, self-sufficient container that will run virtually anywhere.

Docker containers can encapsulate any payload, and will run consistently on and between virtually any server. The same container that a developer builds and tests on a laptop will run at scale, in production*, on VMs, bare-metal servers, OpenStack clusters, public instances, or combinations of the above.

We want to be able to include data similar to this in the container image.  We again take advantage of the LABEL patch, and add data like the following to the Dockerfile.

LABEL Name=apache_php
LABEL Version=1.0
LABEL Vendor=”Red Hat” License=GPLv3
LABLE Description=”
The Apache PHP Application is a example of using atomic command to install a service onto a machine.”

atomic info apache_php
Name         : apache_php
Version      : 1.0
Vendor       : Red Hat
License      : GPLv3
INSTALL      : docker run --rm --privileged -v /:/host -e HOST=/host -e LOGDIR= -e CONFDIR= -e DATADIR= -e IMAGE=IMAGE -e NAME=NAME IMAGE /bin/install.sh
UNINSTALL    : docker run --rm --privileged -v /:/host -e HOST=/host -e IMAGE=IMAGE -e NAME=NAME IMAGE /bin/uninstall.sh

DevNation 2015 – agenda is now posted


Thinking about attending DevNation 2015 in Boston?  The new agenda just got posted.

Reminder: early bird pricing has been extended through April 24th.  If you’re fingers and toes are not available, that’s this Friday.  ;)


More about DevNation:  DevNation 2014, was our inaugural open source, polyglot conference for application developers and maintainers. Learn about DevNation and view its many recorded sessions here.  DevNation 2015 will be in Boston, MA, USA, June 23-26, 2015.  Be sure to follow its status on www.devnation.org.


March in JBoss


A little late for a March review. But March was late in general. The weather didn’t really pick up and we had some scheduling issues on the JBoss weekly editorial. Things like that happen, I guess. Time for me to finally catch up with all the news and releases in March. In the middle of April. My apologies.

March In JBossMore than 20 Releases
Everybody woke up from hibernation and we’ve seen 20+ releases of JBoss Open Source Projects and Products in March. One of the most important ones was the announcement of the PicketLink and Keycloak project merge. And the fact, that Camel version 2.15.0 came out in 2015 is also a nice reason to blog. The Hibernate team keeps their busy schedule and they are releasing monthly. This includes Search and OGM.

Containers And Microservices Are Dominating Middleware
The most prominent topic around middleware are Containers. We’ve seen all kinds of posts around how to use Docker Compose, or Microservices Architectures or even news from Fabric and OpenShift. And we’ve also been talking about the cost of change for microservices architectures. And more and more projects are exploring how to run on OpenShift v3 which will be based on containers and orchestration. And Java EE is all over the place. Arun talks about 9 Docker recipes for Java EE Applications and continues his successful tech-tip series.

JBoss Fuse Workshop by Christina Lin
One very courageous piece of work was started by Christina Lin. She developed a six part workshop series around JBoss Fuse aimed at beginners.

That’s all for today. Make sure to subscribe to the JBoss Weekly Editorial blog and follow @jbossdeveloper or @myfear on Twitter for latest news and updates in Red Hat JBoss Middleware.


You can learn more about Red Hat JBoss Middleware products by visiting: Red Hat JBoss Middleware product pages, and JBoss Community is now Wildfly!