Apache Kafka, Microservices, Kubernetes & Zookeeper

Apache Kafka

Apache Kafka is a distributed publish-subscribe messaging system and a robust queue that can handle a high volume of data and enables you to pass messages from one end-point to another. Apache Kafka is an alternative to a traditional enterprise messaging system.

Apache Kafka can handle millions of data points per second, which makes it well-suited for big data challenges.

Microservices is an architecture and an approach to writing software.

Using microservices, applications are broken down into their smallest components, independent from each other.

Developers need a scalable platform to host Kafka applications, so Kubernetes is the answer.

Kubernetes also makes your development process more agile.

Kubernetes—the technology behind Google’s cloud services—is an open source system for managing containerized applications, and it eliminates many of the manual processes associated with containers.

ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.

 

Kafka, apache zookeeper download

kafka 2.3.0

zookeeper

Advertisement

How to install Go 1.13.3 on CentOS 8/7.x

Go Language:

The Go programming language is an open source project to make programmers more productive.

Main features:

Go compiles quickly to machine code yet has the convenience of garbage collection and the power of run-time reflection

Multicore and Networked machines

Support target: windows 32/64bits, linux 32/64bits, MacOS

IDE Integration with vim, visual studio code, goland plugin (intellij) and atom

Quick Guide:

1. Download using the url: Go 1.13.3 64bits
2. tar xfz
   go1.13.3.linux-amd64.tar.gz -C /usr/local/
3. Added this to your profile
   export PATH=$PATH:/usr/local/go/bin
4. Test your installation by the command: go version
5. Download the extra Lib:
   go get <git_repo>/<your_module>/<your_lib>

   Go Code example: (for diagnostic)
   
   diagnostic.go

package main

import (
“log”
“net/http”
“net/http/pprof”
)

func main() {
mux := http.NewServeMux()
mux.HandleFunc(“/custom_debug_path/profile”, pprof.Profile)
log.Fatal(http.ListenAndServe(“:5555”, mux))
}

Redhat / CentOS 8.0 News and issues

RHEL 8.0 / CENTOS 8.0

 

Main New Features:

• Glibc 2.28, BIND 9.11
• Java 11, icetea-web, ant, maven, scala
• CMake 3.11, Make 4.21, Gcc 8.2, stace 4.24, Go 1.11.5, Binutils 2.30 (optimization for IBM Z)
• GNOME Shell, version 3.28 and GDM use Wayland as default
• Git version changed to 2.18, Mercurial to 4.8 and subversion to 1.10
• Python 3 support, PHP 7.2 support
• Ruby 2.5 support, Perl 5.26 support, Node.js new support
• Apache HTTP Server 2.4.37, nginx web server 1.14, Squid 4.4 and Varnish Cache 6.0
• The nobody user replaces nfsnobody (user and group pair with the ID of 65534)
• PostgreSQL 10, PostgreSQL 9.6, MySQL 8.0, and MariaDB 10.3
• Unicode 11.0.0 support (glibc), GDB debugger version 8.2
• DNF – YUM (v4) support modular content and Increased performance (v4 is compatible with YUM v3)
• CUPS logs are handled by journald (the end of the logs: /var/log/cups)
• Anaconda supports a new kernel boot parameter in RHEL 8, Using RPM interface: 4.14. (old RPM: 4.11)
• XFS now supports shared copy-on-write data extents (Maximum XFS file system size is 1024 TiB)
• ext4 file system now supports metadata checksum
• webconsole with firewall interface, using webconsole by default
• Changed IdM integration (CA, ssh, etc..) with webconsole 
• Added support for mobile browsers in webconsole
• Added support to manage Virtual Machine in webconsole
• Installing RHEL from a DVD using SE and HMC is now fully supported on IBM Z
• Anaconda supports a unified ISO in RHEL 8
• Anaconda can install modular packages in Kickstart scripts
• RHEL 8 supports installing from a repository on a local hard drive
• Added new support for ARM 52-bit 
• Added support to Intel Omni-Path Architecture (OPA)
• Kernel now supports 5-level page tables (4PB of physical memory, 128PB of virtual address space)
• Supports enhanced IBRS for future Intel CPUs
• Active Directory users can now administer Identity Management
• Identity Management packages are available as a module (yum module install idm:DL1/profile_name command)
• Added the 64-bit ARM architecture in toolchain components

Some known issues:

Some extra limitations for Wayland with NVIDIA users:

Multi-GPU setups are not supported under Wayland.
The NVIDIA binary driver does not work under Wayland.
The xrandr utility does not work under Wayland
No clipboard manager is available
Use the X11 session for virtual environments. (Wayland inside guest virtual machines)

workaround: 

Set WaylandEnable=false to the file /etc/gdm/custom.conf

 

Special Warning for CentOS/VirtualBox users

If you are planning to install CentOS-8 in a VirtualBox guest, you should not select “Server with a GUI” (default) during the installation

Installing the VirtualBox Addons will produce an error if your version is 6.0.12/5.2.32 or lower. This is fixed with versions 6.0.14 and 5.2.34

PackageKit is unable to resolve local DNF/YUM variables

CentOS 8 in VirtualBox 5.2.32 (graphical install screen problem):

using the vbox Host key + 2 to switch to VT 2 and then vbox Host key + 6 to switch back to the graphical install screen.

How to install Opera 56 in RedHat 7.6

yum localinstall opera-stable_56.0.3051.116_amd64.rpm

Loaded plugins: langpacks, product-id, search-disabled-repos
Examining opera-stable_56.0.3051.116_amd64.rpm: opera-stable-56.0.3051.116-0.x86_64
Marking opera-stable_56.0.3051.116_amd64.rpm to be installed
Resolving Dependencies
–> Running transaction check
—> Package opera-stable.x86_64 0:56.0.3051.116-0 will be installed
–> Processing Dependency: libXss.so.1()(64bit) for package: opera-stable-56.0.3051.116-0.x86_64
localrh76 | 4.3 kB 00:00:00
–> Running transaction check
—> Package libXScrnSaver.x86_64 0:1.2.2-6.1.el7 will be installed
–> Finished Dependency Resolution

Dependencies Resolved

================================================================================================================================================================
Package Arch Version Repository Size
================================================================================================================================================================
Installing:
opera-stable x86_64 56.0.3051.116-0 /opera-stable_56.0.3051.116_amd64 180 M
Installing for dependencies:
libXScrnSaver x86_64 1.2.2-6.1.el7 localrh76 24 k

Transaction Summary
================================================================================================================================================================
Install 1 Package (+1 Dependent package)

Total size: 180 M
Total download size: 24 k
Installed size: 180 M
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libXScrnSaver-1.2.2-6.1.el7.x86_64 1/2
Installing : opera-stable-56.0.3051.116-0.x86_64 2/2
Redirecting to /bin/systemctl start atd.service
Verifying : opera-stable-56.0.3051.116-0.x86_64 1/2
Verifying : libXScrnSaver-1.2.2-6.1.el7.x86_64 2/2

Installed:
opera-stable.x86_64 0:56.0.3051.116-0

Dependency Installed:
libXScrnSaver.x86_64 0:1.2.2-6.1.el7

Complete!

 

> cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.6 (Maipo)

> opera –version
56.0.3051.116

Download the RPM:

https://www.opera.com/computer/thanks?partner=www&par=id=44510%26location=415&gaprod=opera

Greater Bay Area 粵港澳大灣區(大湾区)

China’s Bay Area Plan 大湾区 –  Mega project of China
(Guangdong-Hong Kong-Macao)

innovation, AI cities and technology hub
(a collective GDP of US$1.5 trillion)

The HK-Zhuhai-Macau mega bridge is part of this project.

11 cities of south of china

香港、澳門
廣州、深圳、珠海、佛山、東莞、中山、江門、惠州和肇慶
Hong-Kong, Macau
Guangzhou, Shenzhen, Zhuhai, Foshan, Dongguan, Zhongshan,Jiangmen, Huizhou

Shenzhen is the leading technology and innovation hub in China.
Macau – AI and Entertainment city
HK – economic and sale’s room city
Dongguan – new IA manipulatory base

粵港澳大灣區總面積約5萬6000平方公里，2016年常住人口約6774萬人，區內生產總值GDP佔全國12%，約1 萬 3879 億美元，人均GDP近2萬500美元。