Microservices architecture is a distinctive method of developing software systems that tries to focus on building single-function modules with well-defined interfaces and operations known as containers. By holding an application’s complete runtime environment, including libraries, binaries, and configuration files, platform and infrastructure are abstracted, allowing the application to run more or less anywhere. The trend has grown popular in recent years as enterprises look to become more agile and move towards a DevOps and continuous testing. The idea behind microservices is to allow developers to build their applications from various independent components which can easily be changed, removed, or upgraded without affecting the whole application. Microservices create scalable, testable software that can be delivered in very short periods of time. Netflix, eBay, Amazon, Twitter, PayPal, and many other tech stars have all evolved from microservices architecture. Containers are available from all the major cloud providers as well as in on-premises data centers and hybrid clouds.
Human language doesn't speak in zeros and ones, but there's a lot of benefit and productivity that can be gained when machines are taught to read, decipher, understand, and make sense of the human language in a manner that is valuable. That's the goal of Natural Language Processing, usually shortened as NLP. Early efforts at this include pieces of digital assistants like Alexa, Microsoft Cortana, Google Assistant, and Siri. It is the driving force behind such common applications as Google Translate, the grammatical checking in Microsoft Word, and Interactive Voice Response (IVR) applications used in call centres. NLP is also essential when it comes to working with many types of unstructured data such as the data in electronic health records, emails, text messages, transcripts, social media posts -- anything with a language component. It's through NLP that we can get to more advanced technologies such as sentiment analysis.
Many different classes of machine-learning algorithms have been applied to natural-language-processing tasks. These algorithms take as input a large set of "features" that are generated from the input data. Thus, NLP has evolved into research focused on statistical models which make soft, probabilistic decisions based on attaching real-valued weights to each input feature. These models have the advantage that they can express the relative certainty of many different possible answers rather than only one, producing more reliable results when such a model is included as a component of a larger system. Systems based on machine-learning algorithms have many advantages and they all are driving NLP forward as a hot skill area to invest in. Consider the following.
- Learning procedures used during machine learning automatically focus on the most common cases, whereas when writing rules by hand it is often not at all obvious where the effort should be directed.
- Automatic learning procedures can make use of statistical inference algorithms to produce models that are robust to unfamiliar input (e.g., containing words or structures that have not been seen before) and to erroneous input (e.g. with misspelled words or words accidentally omitted). NLP’s advantage is that creating systems of handwritten rules that make soft decisions is extremely difficult, error-prone and time-consuming.
- Systems based on automatically learning the rules can be made more accurate simply by supplying more input data. There is a limit to the complexity of systems based on handcrafted rules, beyond which the systems become increasingly unmanageable. But creating more data to input to machine-learning systems simply requires a corresponding increase in the number of man-hours worked, generally without significant increases in the complexity of the annotation process.
The Scaled Agile Framework (SAFe) is a set of organisation and workflow patterns intended to guide enterprises in scaling lean and agile practices. Along with large-scale Scrum (LeSS), disciplined agile delivery (DAD), and Nexus, SAFe is one of a growing number of frameworks that seek to address the problems encountered when scaling beyond a single team. SAFe promotes alignment, collaboration, and delivery across large numbers of agile teams. It was developed by and for practitioners, by leveraging three primary bodies of knowledge: agile software development, lean product development, and systems thinking.
- [Tie] Azure Machine Learning
Redis
Average pay premium: 17% of base salary equivalent
Market value increase: 21.4% (in the six months through July 1, 2022)
Azure Machine Learning (AML) is a cloud-based service for creating and managing machine learning solutions and accelerating the machine learning project lifecycle. It’s designed to help data scientists and machine learning engineers leverage their existing data processing workflows, model development skills and frameworks, and also to help them to scale, distribute, and deploy their workloads to the cloud. Application developers will find AML tools for integrating models into applications or services while Platform developers will find a robust set of tools, backed by durable Azure Resource Manager APIs, for building advanced ML tooling. Users can create a model in Azure Machine Learning or use a model built from an open-source platform, such as Pytorch, TensorFlow, scikit-learn, XGBoost, or LightGBM. Other languages and frameworks are supported as well, including R and .NET.
Redis (for Remote Dictionary Server) is a fast, open source, in-memory, key-value data store. It delivers sub-millisecond response times, enabling millions of requests per second for real-time applications in industries like gaming, ad-tech, financial services, healthcare, and IoT. Because of its fast performance, Redis is a popular choice for caching, session management, gaming, leaderboards, real-time analytics, geospatial, ride-hailing, chat/messaging, media streaming, and pub/sub apps.
Use cases for Redis:
· Real-time data store. Redis' versatile in-memory data structures let you build data infrastructure for real-time applications requiring low latency and high-throughput.
· Caching & session storage. Redis' speed makes it ideal for caching database queries, complex computations, API calls, and session state.
· Streaming & messaging. The stream data type enables high-speed data ingestion, messaging, event sourcing, and notifications.
The benefits of Redis include:
- All Redis data resides in memory, which enables low latency and high throughput data access.
- Replication and persistence. Redis employs a primary-replica architecture and supports asynchronous replication where data can be replicated to multiple replica servers, providing improved read performance and for persistence, point-in-time backups are supported.
- Flexible data structures. Unlike other key-value data stores that offer limited data structures, Redis has a vast variety of data structures to meet your application needs.
- Simplicity and ease-of-use. With Redis, you write fewer lines of code to store, access, and use data in your applications. Developers can use a simple command structure as opposed to the query languages of traditional databases. Over a hundred open source clients are available for Redis developers. Supported languages include Java, Python, PHP, C, C++, C#, JavaScript, Node.js, Ruby, R, Go, and many others.
- High availability and scalability. Redis offers a primary-replica architecture in a single node primary or a clustered topology. This allows you to build highly available solutions providing consistent performance and reliability.
- Open Source. Redis is an open source project supported by a vibrant community, including AWS. There’s no vendor or technology lock in as Redis is open standards based, supports open data formats, and features a rich set of clients.
- [Tie] Big Data
Data Analytics
Data Integration
Informatica
Network Architecture
Neural Networks
Average pay premium: 17% of base salary equivalent
Market value increase: 13.3% (in the six months through July 1, 2022)
But what is Big Data exactly? It can be defined as data sets whose size or type is beyond the ability of traditional relational databases to capture, manage and process the data with low latency. Characteristics of big data include high volume, high velocity and high variety. Sources of data are becoming more complex than those for traditional data because they are being driven by artificial intelligence (AI), mobile devices, social media and the Internet of Things (IoT). For example, the different types of data originate from sensors, devices, video/audio, networks, log files, transactional applications, web and social media — much of it generated in real time and at a very large scale.
Current usage of the term big data tends to refer to the use of predictive analytics, user behavior analytics, or certain other advanced data analytics methods that extract value from big data, and seldom to a particular size of data set. Analysis of data sets can find new correlations to "spot business trends, prevent diseases, combat crime and so on". Scientists, business executives, medical practitioners, advertising professionals and governments alike regularly meet difficulties with large data-sets in areas including internet searches, fintech, healthcare analytics, geographic information systems, urban informatics, and business informatics. Scientists encounter limitations in e-Science work, including meteorology, genomics, connectomics, complex physics simulations, biology, and environmental research.
Data analytics is the process of examining data sets in order to draw conclusions about the information they contain, increasingly with the aid of specialised systems and software. Data analytics technologies and techniques are widely used in commercial industries to enable organisations to make more-informed business decisions, and by scientists and researchers to verify or disprove scientific models, theories and hypotheses. Data analytics initiatives can help businesses increase revenues, improve operational efficiency, optimise marketing campaigns and customer service efforts, respond more quickly to emerging market trends and gain a competitive edge over rivals - all with the ultimate goal of boosting business performance. Depending on the particular application, the data that's analysed can consist of either historical records or new information that has been processed for real-time analytics uses. In addition, it can come from a mix of internal systems and external data sources.
Data integration involves combining data residing in different sources and providing users with a unified view of them. This process becomes significant in a variety of situations, which include both commercial (such as when two similar companies need to merge their databases) and scientific domains (combining research results from different bioinformatics repositories, for example). Data integration appears with increasing frequency as the volume (that is, big data) and the need to share existing data explodes. It has become the focus of extensive theoretical work, and numerous open problems remain unsolved. Data integration encourages collaboration between internal as well as external users.
Informatica is a software development company whose core products include enterprise cloud data management, data integration (extract, transform, load), information lifecycle management, business-to-business data exchange, cloud computing integration, complex event processing, data masking, data quality, data replication, data virtualisation, master data management, ultra messaging, and data governance. It has a customer base of more than 9,500 companies.
In its cloud data marketplace, Informatica offers a cloud-native business user-friendly data search and acquisition facility that uses data intelligence collected and curated by the Intelligent Data Management Cloud platform. Informatica designed it to be a service for sharing data analytics and machine learning and AI models, and this relies on having good datasets and the ability for users to find the right model and relevant data in the marketplace is important for organisations.
Network architecture refers to the way network devices and services are structured to serve the connectivity needs of client devices. It refers to a network’s structural and logical layout and describes how the network devices are connected and the rules that govern data transfer between them. There are many ways to approach network architecture design, which depend on the purpose and size of the network.
Planning the network architecture is vital because it either enhances or hinders the performance of the entire system. Choosing the wrong transmission media or equipment for a particular expected server load, for instance, can cause slowdowns on the network. Network architecture can also facilitate security, which has become increasingly important as more user devices connect to the network. The design and protocols of the network need to support quick and efficient user recognition and authorisation.
Types of Networking Architecture. While there are myriads ways to design a network architecture; most fall into one of two types, the peer-to-peer and client/server architectures. In a peer-to-peer model, all devices in a network have equal responsibilities and privileges with each other as tasks are allocated equally throughout the network. In a client/server architecture, all devices in the network (clients) are connected to a central hub (server) that handles the bulk of the network operations – data storage, processing of client requests, cybersecurity, and access control. There’s also a hybrid architecture---edge computing---which is becoming more popular with the Internet of Things (IoT). It’s similar to a client/server architecture however instead of the server being responsible for all storage and processing tasks, some of it is delegated to computers located closer to the client machine, called edge devices.
Network Architecture Design. The design of any digital network architecture involves optimising its building blocks. These include: