IT Open-researching ecosystem
KEXEntrance, Since 2022
By @DuyDA | Infrastructure Engineer
DIGI-TEXX's first IT R&D Infra. From & for DIGI-TEXX
About KEXENtrance PROJECT
More than a platform!
KEXEntrance project is oriented to be an Open-researching ecosystem, fueled by non-stop learning mindset, a home of tech-lovers, IT engineers. A reliable source of experience sharing needs a "technical leader", which is the heart of Development & Operation, KEXEntrance platform
Embracing $BASH automation and other supportive programming language (R, Python, NodeJS,...) in agile designs, KEXEntrance is a command-line-based web-support platform. Using Secure Shell (ssh) & Hypertext Transfer Protocol Security (https) protocols for connectivity & data transmission. Which platform is oriented to be an Infrastructure as code solution.
The Core of platform is designed with built-in command line interface (KEXcli, KEXcla), scripts and modulars to bring convenience & effectiveness in IT system administrations. There are generated public & private keypairs to authorized communication(s)/connection(s) between KEXEntrance and the rest system in DIGI-TEXX's IT environment (servers, workstations,...).
HIGHLIGHTS
CLI-aliasing for Linux OS & Sys-protect solutions
CLI-aliasing for Linux OS & Sys-protect solutions
CLI-aliasing for Linux OS & Sys-protect solutions
A stack of automations help default OS CLI of Linux Machine(s) in KEXEntrance Ecosystem to be transformed into "encoded" form, that even the local ROOT account(s) of Linux Machines(s) can not perform executions.
Cybersecurity topics are important!
Linux environment is secured by default, is not friendly for unauthorized outsiders to exploi
A stack of automations help default OS CLI of Linux Machine(s) in KEXEntrance Ecosystem to be transformed into "encoded" form, that even the local ROOT account(s) of Linux Machines(s) can not perform executions.
Cybersecurity topics are important!
Linux environment is secured by default, is not friendly for unauthorized outsiders to exploit the attacks.
However, our topic is more than being a attacked scenario. During the system managements & operations, there are reasons to have Linux OS default Command Line Interface (CLI) to be filtered/restricted even with "Root" or user with "Sudoer" permission. These can be important, sensitive and easy to use commands like "passwd" (changing system user password), "rm" (deleting system data), "init" (initializes and controls processes, reboot / shutdown the system)
Here is some "Why?"
First: Buggy new deployed executable scripts or accidental unexpected actions by server administrator can put the system in bad situation
Second: Supporting the permission control with care. In the large scale IT environment, there are many servers and systems. Sometime, there is mistaken permission granting, to an IT intern for example. Without deep knowledge in system management, he can potentially harm the system.
Third: Even when attackers or virus can get into the system somehow, bypassing all the other security level, there will NOT be a chance for them to make any changes to the system.
System DataBot with Distributed key-value store
CLI-aliasing for Linux OS & Sys-protect solutions
CLI-aliasing for Linux OS & Sys-protect solutions
KEXEntrance improves the consistency in storing data, which proves to be the trustworthy source for the systems to connect and use reliably.
ETCD with RAFT algorithm is the most suitable High Availability solution to integrate, result in the born of a useful native feature called KEXEntrance-dataBot.
ETCD is a distributed system. It provid
KEXEntrance improves the consistency in storing data, which proves to be the trustworthy source for the systems to connect and use reliably.
ETCD with RAFT algorithm is the most suitable High Availability solution to integrate, result in the born of a useful native feature called KEXEntrance-dataBot.
ETCD is a distributed system. It provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines.
While KEXEntrance repository (KEXrepo) provides types of data, ETCD cluster(s) in KEXEntrance is specialized contains configuration files (*.conf), configuration guideline & KEXEntrance platform scripts.
Servers in the KEXEntrance ecosystem can get data from the ETCD cluster easily with "etcdctl" command
For example, you want to build a Linux Samba fileserver with 3 simple steps:
1. Download & install samba packages from KEXrepo.
2. Get the configuration file, configuration guideline from the ETCD cluster.
3. You are ready to go, pretty much simple.
Kex.devtool, platform developer tool
CLI-aliasing for Linux OS & Sys-protect solutions
Kex.devtool, platform developer tool
Tool for developer is essential of any technical platform as the development source is scaling up quickly. Hence, kex.devtool is rolled out for person(s) who at least be the platform Developer to use. This is a dedicated designed native tool which plays the main role in ensuring numerous scripting source, that can be tracked and being kep
Tool for developer is essential of any technical platform as the development source is scaling up quickly. Hence, kex.devtool is rolled out for person(s) who at least be the platform Developer to use. This is a dedicated designed native tool which plays the main role in ensuring numerous scripting source, that can be tracked and being kept developing conveniently.
Moreover, kex.devtool supports mirroring KEXEntrance's source code back and forth between local filesystem of KEXEntrance platform & ETCD cluster in the ecosystem.
With kex.devtool, not only managing platform's sources of development turns easy, but also further integrations as KEXEntrance Ecosystem keeps growing.
The "robotic army"
Integrated-driven Ecosystem
Kex.devtool, platform developer tool
@KEXAgent(s) are minions of KEXEntrance platform, are designed to be as Linux Systemd service(s), to be installed directly onto the remote linux device(s) for central management, integration, deployment activities.
There will be definitely also many future forms of @agents to work well on other types of OS like Windows, MacOS,...v...v... B
@KEXAgent(s) are minions of KEXEntrance platform, are designed to be as Linux Systemd service(s), to be installed directly onto the remote linux device(s) for central management, integration, deployment activities.
There will be definitely also many future forms of @agents to work well on other types of OS like Windows, MacOS,...v...v... But the start is Linux
These @agents create a Command-waiting network. They maintain the closely links to a functional zone on KEXRepo (The special web storage of the platform) where the "commander" of these @agents can define what are going to be done next before the entire Command-waiting network starts executions.
KEXImage-2-Filesystem
Integrated-driven Ecosystem
Integrated-driven Ecosystem
A special designed stack of automations that allows KEXEntrance platform to distribute KEXImage(s) to servers, workstations,... via iSCSI connections.
These Images are pre-encrypted with LUKS standard before distribution and then be formatted by a filesystem type to contain data as common.
The KEXImage's operation is flexible while the tota
A special designed stack of automations that allows KEXEntrance platform to distribute KEXImage(s) to servers, workstations,... via iSCSI connections.
These Images are pre-encrypted with LUKS standard before distribution and then be formatted by a filesystem type to contain data as common.
The KEXImage's operation is flexible while the total capacity of those can be increased & be decreased easily by the autonomous handler, base on the real usage, avoid containing sparse data troubles causing waste situations in storage system management.
Integrated-driven Ecosystem
Integrated-driven Ecosystem
Integrated-driven Ecosystem
As an ecosystem, KEXEntrance ensures its native platform can be cross-working good with other IT products to increase the effectiveness in many manners. On the other hands, proving its independence with self-designed features from scratch!
1. A domain-controller-like power: KEXEntrance gives out a unified User Credential Base for authentic
As an ecosystem, KEXEntrance ensures its native platform can be cross-working good with other IT products to increase the effectiveness in many manners. On the other hands, proving its independence with self-designed features from scratch!
1. A domain-controller-like power: KEXEntrance gives out a unified User Credential Base for authentications within its under-managed "terrotories"
2. Third product integrations: Our autonomous platform contains higher level stacks of automations help it can "speak to" and work well with other platform-based or utility-based products in the market that are easy to use & are directly deployed in Production IT Infrastructure
Typically as a KEXcli stack for autonomous managemennt process of XCP-ng environment, a community powered virtualization product. KEXEntrance first stone is laid on DIGI-TEXX's XCP-ng platform in early 2022 which is farmilar enough to have the integration of KEXEntrance core and Xen Orchestra is not a difficult manner.
3. Platform's IDE: A programming factory, supports multi programming languages for R&D and training required KEXapps, APIs.
4. R&D Project: KEXEntrance manages more complex open source / commercial systems, which are harder to master, to control.
Those will be kept running to continue theirs developments and optimizations, whether they are deployed or not yet been deployed onto IT Production Infrastructure.
These systems are designed and built during activities in R&D Infrastructure with multiple test scenarios to evaluate and report to management level (IT Manager, CTO).
Multi Factor Authentication (MFA)
Advanced Message Queuing Protocol (AMQP)
Advanced Message Queuing Protocol (AMQP)
The best way to prevent bad actor to have chance attacking your system is implementing the Multi Factor Authentication (MFA) model.
KEXEntrance platform is designed to mainly manage systems via SSH. By default, IT engineers can use username and password for common authentication. To enhance the security level, SSH keypair (public & private
The best way to prevent bad actor to have chance attacking your system is implementing the Multi Factor Authentication (MFA) model.
KEXEntrance platform is designed to mainly manage systems via SSH. By default, IT engineers can use username and password for common authentication. To enhance the security level, SSH keypair (public & private) can be configured.
Or 3rd application like Google Authenticator can also be added as the extra layer in authentication
The default implementation of this feature in KEXEntrance ecosystem is the communication between the Open Web Console and the KEXEntrance Executor upon the activation of every authorized account.
Advanced Message Queuing Protocol (AMQP)
Advanced Message Queuing Protocol (AMQP)
Advanced Message Queuing Protocol (AMQP)
This message-oriented middleware is an essential implementation in an IT environment with microservices software architecture.
RabbitMQ is a popular being used message-broker software to stay this intermediary.
RabbitMQ can be easily deployed as Cluster(s) in Docker-based environment under container form. Being managed by DevOps engineer,
This message-oriented middleware is an essential implementation in an IT environment with microservices software architecture.
RabbitMQ is a popular being used message-broker software to stay this intermediary.
RabbitMQ can be easily deployed as Cluster(s) in Docker-based environment under container form. Being managed by DevOps engineer, supporting Developers delivering software to consumer effectively.
Containerization
Advanced Message Queuing Protocol (AMQP)
Containerization
Docker!
An open platform for developing, shipping and running applications.
Docker is also known as OS-level virtualization for delivering software packages called containers.
While a Virtual machine (VM) is provided by virtualization technology lets you run a virtual machine on any hardware. Docker lets you run an application on any oper
Docker!
An open platform for developing, shipping and running applications.
Docker is also known as OS-level virtualization for delivering software packages called containers.
While a Virtual machine (VM) is provided by virtualization technology lets you run a virtual machine on any hardware. Docker lets you run an application on any operating system. In other words, if considering in IT Infrastructure build:
- Hardware as layer 1,
- Virtualization (Host-OS) as layer 2,
- Guest-OS (Virtual Machine) as layer 3, => We have Docker contains software packages (containers) is layer 4.
Moreover, Containers have near-instant startup times because they share the host OS's kernel and do not need to boot a full OS like a VM.
So, yes Docker is "lightweight", is more suitable for agile project deployment that does not requires steadiness with less architecture changes like virtualization platforms in IT Infrastructure
CI/CD pipeline
Public Key Infrastructure (PKI)
Containerization
A CI/CD pipeline is a series of steps that streamline the software or script delivery process. Via a DevOps or site reliability engineering approach, CI/CD improves app development using monitoring and automation.
Gitlab CI/CD platform utilization is for providing a great integrative coordination with KEXEntrance platform. Where Gitlab-ru
A CI/CD pipeline is a series of steps that streamline the software or script delivery process. Via a DevOps or site reliability engineering approach, CI/CD improves app development using monitoring and automation.
Gitlab CI/CD platform utilization is for providing a great integrative coordination with KEXEntrance platform. Where Gitlab-runner(s) can obtain the available SSH keyvault as the environment to easily deliver the deployment into multiple target systems.
Gitlab CI/CD is also an important part in joining hands to build KEXEntrance repository for the R&D infrastructure.
Object Storage integration
Public Key Infrastructure (PKI)
Public Key Infrastructure (PKI)
API compatible with the Amazon Simple Storage Service (Amazon S3).
KEXEntrance integrates MinIO into the R&D infrastructure repository development.( KEXEntrance repository )
As a centralized fileserver of the entire ecosystem, KEXEntrance repository contains big amount of data in types. Before being integrated with S3-like service like MinI
API compatible with the Amazon Simple Storage Service (Amazon S3).
KEXEntrance integrates MinIO into the R&D infrastructure repository development.( KEXEntrance repository )
As a centralized fileserver of the entire ecosystem, KEXEntrance repository contains big amount of data in types. Before being integrated with S3-like service like MinIO, the repo strongly supports multiple system-level protocols for other authorized factor to connect and use, such as SMB/CIFS, NFS, iSCSI.
Especially, with the supports of MinIO SDK, it is a great condition for KEXEntrance to impoves the Python-based inner-apps in data management
Public Key Infrastructure (PKI)
Public Key Infrastructure (PKI)
Public Key Infrastructure (PKI)
A Cyber Security move to actively defense system against man-in-the-middle (MITM) attack
Not all the cases are requires organization's global certificates in system configuration, which is paid to have from global CA. R&D or deeper engineering activities for example.
KEXEntrance is also building a powerful interal root Certificate Authorit
A Cyber Security move to actively defense system against man-in-the-middle (MITM) attack
Not all the cases are requires organization's global certificates in system configuration, which is paid to have from global CA. R&D or deeper engineering activities for example.
KEXEntrance is also building a powerful interal root Certificate Authority (CA) server for signing certificates. A strategy to support SSL/TLS in product/system communication without relying on 3rd party. E.g: TLS communication between Daemons of Server & Client in Bacula system.
Moreover, PKI indicates how we manage the authorizations of public keys in the IT environment. As KEXEntrance platform is also jumping to manage servers/workstations via SSH key pairs (private & public), the implementation of a real PKI is essential for a great key vault management.
Development pipeline
Time flies, passing significant milestones corresponding with daily primary IT operation and development of Infrastructure Engineering team at DIGI-TEXX. KEXEntrance gradually runs into its shape form and give thes best support to the effectiveness of the IT Infrastructure management.
The platform roles include roles central secured server access gateway, system configuring, standardizing, process automating, technical web storage, drive, engineering knowledge base, solution factory and more...
Recent event(s) that are remarkable:
- 22 June 2024: KEXEntrance's public landing page is inspired and is rolled out officially on 24 June 2024, under kexentrance.com domain.
It is noteworthy that 22 June is also @DuyDA's birthday, In the name of @DuyDA's technical blog to share his personal experiences and unique achievements in his IT career, this landing page is a true remarkable gift for future technical growth inspiration.
- 1 July 2024 -> 08 July 2024: The birth story of R&D Hands-on lab & Sandbox (Details can be found in FAQ section ).
By building the network gateway on company's GUEST network. R&D infrastructure is physically isolated from company's Production Infrastructure. This is the remarkable milestone for the dedication of tech innovation & testing activities in a safe box
- August 2024: During the security feature development in source code of KEXEntrance platform. @DuyDA indicates a possibility of security finding which is strongly harmful to DIGI-TEXX if it is known by bad actors. @DuyDA raises the finding to company's Chairman and Board of Management, the issue is fixed right after days.
- 5 September 2024: The integration of ETCD helps the KEXEntrance environment be in HA mode. With the distributed key-value store solution, environment's source code, standard configuration files, ... are protected within the ETCD cluster
- 13 September 2024: New native feature called KEXEntrance-dataBot is born with the first technical output, embracing the utilization of ETCD in the cosystem. Help store, update-from types of data reliably has never been so convenient
- 1 October 2024: The final version of the KEXEntrance PowerPoint presentation has been submitted to the DIGI-TEXX Board of Management in preparation for the showcase on 04 October 2024. This marks a significant milestone following the initial platform demo with the Chairman in June 2024.
- 15 October 2024: After reviewing @DuyDA's contributions and sharing several notable strategies at DIGI-TEXX with Chairman and gets his positive feedback on 10 October 2024, including:
1. Leading the planning and implementation of live battery replacements for Emerson UPS in emergency situations that left the entire Data Center unprotected in his first year at the company, which he directly proposed to you and participated in the deployment plan for APC smart UPS for the company’s data center.
2. Being a pioneer engineer in building the open-source Xen server system (XCP-ng) in a context where the company faced overlaps in resource server allocation, with no separate environment for UAT and SIT software.
3. Leading the development of the first site-to-site backup strategy for the company between Anna Building and QTSC data center, integrating the Acronis Cyber Cloud solution.
4. Being the lead engineer in researching, understanding, and successfully building Virtuozzo ( Hyperconverged infrastructure ), DIGI-STACK under the guidance and planning of CTO.
@DuyDA has taken the initiative to propose himself for the position of Digital Transformation Lead at DIGI-TEXX and is currently in the consideration process. Hopefully, his proposal will be approved.
R&D Infrastructure, Hands-on Labs and Sandbox
No interaction with current Production environment. July 8 2024, R&D Infrastructure is planned to be built for the first time with no investment for approaching new technology, IT solutions before further Production integration
"Utilizing what we have" for the first days is the right way to go.
The challenge is accepted with difficulties in hardware, separated network with simply a wifi modem for In
Nested virtualization!
A combination to have the Master Machine for Admin management with GUI and its resource for first virtualization node in the conditional of "starting small".
XCP-ng, Xen Cloud Platform Next Generation plays primary role as an open source virtualization environment for containing KEXEntrance and every other Virtual Machines in further R&D activities
R&D repository ( KEXEntrance Repository version 03 )
R&D Infrastructure's goal is to deliver engineering experience, template system, template solution stacks. These standardized build for engineer to deploy easily.
The technical diagrams stored in R&D Infrastructure are closely match to the provided template to lookup. From the origins, IT Engineer can customized the system base on their practical
No dedicated networking devices like router, firewall. A small-form-factor computers and barebone computer kits designed by Intel is selected to be the gateway for building start of R&D infrastructure.
By connecting to the Guest Wireless network, the gateway allows IP forwarding from the system in R&D Infrastructure, which are linked to the gateway network port via the designed wired network.
R&D I
The R&D labs start being built from scratch. Creativity for the base platform is a must to provide the best condition in the future R&D activity.
Environment setup includes:
- Ubuntu as network gateway
- Oracle VirtualBox, Virtualization Type 2 & Nested XCP-ng 8.2.0 on Windows Desktop for general administration
- XCP-ng 8.3.0, Virtualization Type 1 for primary platform running R&D virtual machine
- Xen
Limited in hardware selection and building condition lead to the creativity in infrastructure designs. The future diagram may have its improvement and change, but the original concept with essential components will be ensured
Working for
ADMIN & CONTRIBUTOR
.png/:/cr=t:10.38%25,l:0%25,w:100%25,h:79.23%25/rs=w:515,h:234,cg:true/fx-gs){kind=logo}
IT ENGINEERING HUB
Every tech-lover is welcome here!
The more you understand the ecosystem, the more you will love it
New feature: KEXEntrance-dataBot by implementing the ETCD cluster as launchpad. More: https://www.linkedin.com/posts/kexentrance_base-on-the-power-of-etcd-clustering-https-activity-7240282553763438595--w31?utm_source=share&utm_medium=member_desktop