[TALK]@Telematika

scalable

Architecting a text analytics system in the cloud

January 2, 2020

Arun Veettil shares his experience and lessons learned developing a customized, enterprise-level NLP platform to replace a leading text analytics vendor platform.

Bringing deep learning into big data analytics using BigDL

January 2, 2020

Xianyan Jia and Zhenhua Wang explore deep learning applications built successfully with BigDL. They also teach you how to develop fast prototypes with BigDL's off-the-shelf deep learning toolkit and build end-to-end deep learning applications with flexibility and scalability using BigDL on Spark.

Network Automation with Salt and NAPALM

January 2, 2020

It is already well known that Salt is a very powerful and flexible system. At CloudFlare it also proved to be extremely scalable, being able to manage hundreds of …

Deploying a scalable JupyterHub environment for running Jupyter notebooks

January 1, 2020

Jupyter notebooks provide a rich interactive environment for working with data. Running a single notebook is easy, but what if you need to provide a platform for many users at the same time. Graham Dumpleton demonstrates how to use JupyterHub to run a highly scalable environment for hosting Jupyter notebooks in education and business.

Distributed real-time highly available stream processing

January 1, 2020

Yu-Xi Lim and Michal Wegrzyn outline a high-throughput distributed software pattern capable of processing event streams in real time. At its core, the pattern relies on functional reactive programming idioms to shard and splice state fragments, ensuring high horizontal scalability, reliability, and high availability.

From Kafka to BigQuery: A guide for delivering billions of daily events

January 1, 2020

What are the most important considerations for shipping billions of daily events to analysis? Ofir Sharony shares MyHeritage's journey to find a reliable and efficient way to achieve real-time analytics. Along the way, Ofir compares several data loading techniques, helping you make better choices when building your next data pipeline.

R you ready for the cloud? Using R for operationalizing an enterprise-grade data science solution on Azure

December 30, 2019

R has long been criticized for its limitations on scalable data analytics. What's needed is an R-centric paradigm that enables data scientists to elastically harness cloud resources of manifold computing capability for large-scale data analytics. Le Zhang and Graham Williams demonstrate how to operationalize an E2E enterprise-grade pipeline for big data analyticsall within R.

An architecture for merging fast data and enterprise applications: The SMACK stack

December 28, 2019

Big data architectures and enterprise/microservice architectures are slowly converging. Big data is transitioning to "fast data," emphasizing streaming over batch processing, while data processing is growing ubiquitous. Dean Wampler explores the SMACK stackSpark, Mesos, Akka, Cassandra, and Kafkaand explains how it addresses the needs of both fast data and the enterprise.

Application caching at Netflix: The hidden microservice

December 28, 2019

Netflix is well known for championing the microservice model, but within the complex layers of dependencies is a hidden service: the caching layer. Scott Mansfield explains how the EVCache service fits into the Netflix experience and how it works in the dynamic cloud environment to provide fast and scalable application data caching.

Microservice standardization

December 26, 2019

Microservice architecture brings freedom for developers, but building a sustainable microservice ecosystem requires holding microservices to high architectural and operational standards. Susan Fowler introduces a set of standards that apply to all microservicesstandards that ensure microservice availability while preserving developer freedom.

Addressing Challenges in an Anycast CDN

December 25, 2019

Performance of online applications directly impacts user satisfaction. A major component of the user-perceived performance of the application is the time spent in …

Developing a secure and scalable frontend

December 25, 2019

As traffic to websites and web applications increases, infrastructure must be put in place to handle scalingbut with that comes an increased risk for security breaches. James Baker and Mira Thambireddy dive into specific client-side vulnerabilities, discussing design patterns that scale an application securely and which frameworks currently in the market already employ these practices.

Segment Routing in Datacenter

December 25, 2019

Segment Routing is a network technology that proposes to solve many of the limitations of traditional IP & MPLS networks. It's proposes to be the bridge between th …

Scalable threat modeling with risk patterns

December 24, 2019

Current approaches to threat modeling emphasize manual analysis by trained teams, which can result in a bottleneck in the development process, reducing the appeal of performing this activity. Stephen de Vries presents a technique that uses reusable risk patterns to open the door to automated and scalable threat modeling.

The wonderful world of webhooks

December 16, 2019

Webhooks allow our applications to exchange data as soon as it happens rather than polling using APIs. Lorna Mitchell covers creating, consuming, and deploying webhooks in a modern, microservices world.

Big data meets the IoT

December 15, 2019

The IoT is fundamentally transforming industries and reconfiguring the technology landscape, but challenges exist for enterprises to effectively realize the value from this next wave of information and opportunity. Cheryl Wiebe explores how leading companies harness the IoT by putting IoT data in context, fostering collaboration between IT and OT and enabling a new breed of scalable analytics.

Building a modern data architecture

December 14, 2019

When building your data stack, the architecture could be your biggest challenge. Yet it could also be the best predictor for success. With so many elements to consider and no proven playbook, where do you begin to assemble best practices for a scalable data architecture? Ben Sharma offers lessons learned from the field to get you started.

How the largest US healthcare dataset in Hadoop enables patient-level analytics in near real time

December 12, 2019

The need to find efficiencies in healthcare is becoming paramount as our society and the global population continue to grow and live longer. Navdeep Alam shares his experience and reviews current and emerging technologies in the marketplace that handle working with unbounded, de-identified patient datasets in the billions of rows in an efficient and scalable way.

Lessons learned running Hadoop and Spark in Docker

December 11, 2019

Many initiatives for running applications inside containers have been scoped to run on a single host. Using Docker containers for large-scale environments poses new challenges, especially for big data applications like Hadoop. Thomas Phelan shares lessons learned and some tips and tricks on how to Dockerize your big data applications in a reliable, scalable, and high-performance environment.

Winning with data: How ThredUp, Twilio, and Warby Parker use data to build advantage

December 7, 2019

Daniel Mintz dives into case studies from three companiesThredUp, Twilio, and Warby Parkerthat use data to generate sustainable competitive advantages in their industries.

How NTSB air disaster analysis can help you in an emergency

December 2, 2019

Matt Rogish explains how NTSB investigations of air disasters have dramatically improved flight safety and applies lessons learned in disaster recovery and analysis, teamwork, task saturation, and systems design to modern software application and infrastructure architecture at scale to achieve higher availability, reduced errors, and more scalable systems.

Building a system that never stops: New Relic at scale

November 27, 2019

New Relic receives over half a trillion data points from customers that need to be processed, stored, and made available ASAP. Matthew Flaming explains why moving from a single application performance product to querying more than a billion events a second to serve multiple products has meant radically reinventing and re-architecting the whole platform multiple times.

Whats next for music services? The answer is in the data

November 16, 2019

Can our real-time distributed data systems help predict whether high-resolution audio is the future of digital music? What about content curation? Paul Shannon and Alan Hannaway explore the future of music services through data and explain why 7digital believes well-curated, high-resolution listening experiences are the future of digital music services.

Becoming friends with Cassandra and Spark

November 15, 2019

Jon Haddad and Dani Traphagen explore all the basics youll need to become best buds with the radically scalable, always-on, and increasingly popular Apache Cassandra database. But wait, there's more. Jon and Dani also cover using Apache Spark for large-scale data processing. What else do you need to know to make new friends?

Blocks in containers: Lessons learned from containerizing Minecraft

November 15, 2019

Its clear containers are here to stay for building reliable, scalable applications. But what about applications developed prior to the advent of containerization? Using Minecraft as her example, Julia Ferraioli explains why retrofitting technology for containers poses conceptual and practical challenges that require an approach different than starting container native.

Concurrency + Distribution = Scalability + Availability: A journey architecting Erlang systems

November 14, 2019

Erlang/OTP has for years been described as the secret sauce to writing scalable and available systems. However, Erlang alone will not perform any magic. Francesco Cesarini walks attendees through the topology patterns of modern distributed Erlang architectures, discussing the trade-offs in scalability and reliability, the actor model, the failure model, and the built-in distribution model.

Deployment and orchestration at scale with Docker Swarm

November 13, 2019

Jrme Petazzoni and AJ Bowen demonstrate building an app from development to production with Docker. Jrme and AJ run a sample app on a single node with Compose and add scaling and load balancing. They then provision a Swarm cluster with Docker Machine and implement multihost communication with overlay networking. The result will be a highly available, scalable deployment for the application.

Designing and architecting for scalability with Erlang/OTP

November 13, 2019

You need to implement a fault-tolerant, scalable, soft, real-time system with requirements for high availability. It has to be event driven and react to external stimulus, load, and failure. It must always be responsive. Francesco Cesarini outlines the Erlang/OTP approach to architecting a distributed system, breaking down the task into 10 steps that can be applied in other languages as well.