Google Cloud Pub/Sub: Asynchronous Messaging Service

Google Cloud Pub/Sub
Google Cloud

Share Post Now :

HOW TO GET HIGH PAYING JOBS IN AWS CLOUD

Even as a beginner with NO Experience Coding Language

Explore Free course Now

Table of Contents

Loading

Google Cloud Pub/Sub is an asynchronous messaging service that helps in decoupling services that produce events from services that process events.

This blog covers an outline of cloud pub/sub including its key features, working, Pub/Sub Lite, pricing, etc.

What Is Cloud Pub/Sub?

Google Cloud pub/sub icon

Cloud Pub-Sub is a real-time messaging service offering by Google for applications to publish and subscribe to events.

Pub/Sub= Publisher-Subscriber

One can publish the message manually or watch the changes in Gmail using Gmail API and notify the user application on a real-time basis.

Core Components of Google Cloud Pub/Sub

Pub/Sub basics

  • Topic: A named resource (which acts as an alias) to which the publishers send messages.
  • Publisher: It is an application that creates and sends a message to a Pub/Sub topic. It is additionally referred to as Producer.
  • Subscriber: It is an application that registers itself to a specified topic of interest in order to get appropriate messages.
  • Subscription: It represents a stream of messages from a single, specific topic, which is to be delivered to the subscribing application.
  • Message: It is a combination of data and other attributes which is sent by a publisher to a topic and is delivered to the subscribers at the end.
  • Message attribute: A key-value pair used by a publisher to define a message.

Also, Check Our blog post on the Google Cloud Account Free Trial. Click here

How Does Pub/Sub Works?

The diagram below explains the key components in the Google Cloud Pub/Sub messaging system and the flow of messages between publishers and subscribers.

 

Pub/Sub message flow

  1. A publisher application creates a topic in the Publisher/Subscriber service and sends messages to the topic. The message comprises a payload and optional attributes describing the payload content.
  2. The service makes sure that published messages are retained on behalf of the subscriptions. The message published is retained for a subscription till it is acknowledged by a subscriber consuming messages from that particular subscription.
  3. Pub/Sub forwards messages individually from a topic to all of its subscriptions.
  4. A subscriber receives messages either by Pub/Sub pushing them to the subscriber’s chosen endpoint or by the subscriber pulling them from the service.
  5. An acknowledgment is sent by the subscriber to the Pub/Sub service for each received message.
  6. The service then removes acknowledged messages from the subscription’s message queue.

Also, Read Our previous blog post on Google Cloud Run. Click here

Key Features

1.) In-order delivery at scale: Optional per-key ordering simplifies stateful application logic without sacrificing horizontal scale (i.e., no partitions required).
2.) Stream analytics and connectors: The native Dataflow integration allows reliable, expressive, processing and integration of event streams in languages such as Java, Python, and SQL.

3.) Cost-optimized ingestion with Pub/Sub Lite: Pub/Sub Lite aims to be the lowest cost option for high-volume event ingestion. It offers zonal storage and users have control of capacity management.

4.) Google Cloud–native integrations: Take advantage of integrations with multiple services, such as Cloud Storage and Gmail update events and Cloud Functions for serverless event-driven computing.

Google Cloud Pub/Sub integration

5.) Dead letter topics: It allows for messages unable to be processed by subscriber applications to be put aside for offline examination and debugging so that other messages can be processed without any delay.

Check Out: Cloud Spanner vs Cosmos DB, to know the major differences between them.

Publisher-Subscriber Patterns

The publisher application creates and sends messages to a topic and the subscriber applications create a subscription to that topic to receive messages from it.

There are three pub/sub patterns:

1.) ONE-TO-MANY/Fan-Out

One-to-many pub/sub pattern

2.) MANY-TO-ONE/Fan-In

many-to-one Google Cloud Pub/Sub pattern

3.) MANY-TO-MANY

Many-to-many

Common Use-Cases

Here are some common use cases of Pub/Sub service:

  • Balancing workloads in network clusters
  • Implementing asynchronous workflows
  • Distributing event notifications
  • Refreshing distributed caches
  • Data streaming from various processes or devices
  • For reliability improvement

Read More: About Google Cloud Services. Click here

Pub/Sub vs Pub/Sub Lite

In a Pub/Sub system, publisher applications send messages to topics, and subscriber applications receive messages from subscriptions. Topics and subscriptions can be in different projects as they are global resources.

Pub/Sub Lite is a high-volume message service built for a low cost of operation by offering zonal storage and pre-provisioned capacity. Here, publishers send messages to Lite topics and subscribers receive messages from Lite subscriptions. Lite topics and Lite subscriptions must be in the same Cloud project and zone as they are zonal resources.

Pub/Sub vs Pub/Sub Lite

Pricing

Pub/Sub pricing is based on:

  • Message ingestion and delivery
  • Seek-related message storage i.e., snapshots and retained acknowledged messages

Pub/Sub pricing is based upon the monthly data volumes consumed. The first 10 GB of data per month is offered at no charge and beyond 10GB price is $40 per TB (tebibyte, or 240 bytes).

Read More: About GKE Cluster. Click here

Frequently Asked Questions

What is a pub/sub subscription?

The subscription connects the topic to a subscriber application that receives and processes messages published to the topic. A topic can have multiple subscriptions, but a given subscription belongs to a single topic.

Are messages delivered in order?

The Pub/Sub service can deliver messages in the order if the messages received have the same ordering key and publishers send the messages to the same region,

How to detect duplicate messages?

Pub/Sub assigns a unique message_id to each message, which is used to detect duplicate messages received by the subscriber.

Can Pub/Sub be used for cross-region or cross-project communication?

Yes, Pub/Sub supports cross-region communication within the same project and even across projects. You can publish messages to topics in one region and subscribe to those topics from subscribers in different regions or projects.

How does Pub/Sub guarantee message delivery?

Pub/Sub guarantees at-least-once delivery, which means that messages are delivered to subscribers at least once. If a subscriber acknowledges a message, Pub/Sub removes it from the system. If a message acknowledgment is not received within a certain time, Pub/Sub retries delivery.

How does Pub/Sub work?

Pub/Sub uses a publish-subscribe model, where publishers send messages to specific topics, and subscribers receive and process messages from those topics. Topics act as communication channels, and subscribers can receive messages in real-time or at their own pace.

Related References

Next Task For You

If you are also interested and want to know more about the Google Professional Cloud Architect certification then register for our Free Class.

Picture of mike

mike

I started my IT career in 2000 as an Oracle DBA/Apps DBA. The first few years were tough (<$100/month), with very little growth. In 2004, I moved to the UK. After working really hard, I landed a job that paid me £2700 per month. In February 2005, I saw a job that was £450 per day, which was nearly 4 times of my then salary.