How Does A Multimodal LLM Work? The Imaginative and prescient Story

Characteristic	Express Alignment	Implicit Alignment
Nature	Supervised / Annotated	Unsupervised / Realized throughout coaching
Want for Labels	Requires aligned or annotated knowledge	Doesn’t require specific alignments
Method	Handbook or rule-based mapping	Realized through consideration or contrastive loss
Instance Duties	Picture captioning with bounding containers	CLIP, VQA with unsupervised consideration
Benefits	Excessive precision, interpretable	Scalable, versatile, study fine-grained hyperlinks
Challenges	Costly to label, much less versatile	Could be much less interpretable, data-hungry

We’ll now attempt to perceive one other essential time period that we used within the above part named “fusion” subsequent.

If you wish to perceive how implicit alignment might be accomplished, learn this. On this analysis paper, the mannequin embeds fragments of photos (objects within the picture) and fragments of sentences (typed dependency tree relations) into a standard house.

Let’s dive a bit deeper into this.

The Idea of Fusion in Multimodal LLMs

The cornerstone of multimodal studying lies in understanding how several types of knowledge might be mixed successfully. In different phrases, it serves as a solution to precisely align our completely different modalities throughout a unified dimensional house. Fusion methods decide when and the way data from completely different modalities is built-in, essentially shaping the mannequin’s potential to grasp advanced multimodal inputs.

Fusion refers back to the integration of knowledge from a number of modalities akin to textual content, picture, and audio right into a unified illustration. It performs a important function in enabling fashions to leverage complementary data from every modality. The aim is to mix options in such a approach that the mannequin could make extra knowledgeable predictions. It’s fairly much like the idea of fusion that we use in Deep Studying.

There are two broadly used methods for fusion: Early Fusion and Late Fusion.

There additionally exists a 3rd class – mid-fusion, about which I’ll clarify shortly.

1. Early Fusion

Early Fusion represents the best method to multimodal integration, right here the uncooked knowledge from completely different modalities is mixed on the enter degree itself earlier than any processing happens. In early fusion methods, knowledge from varied sources akin to pixel values from photos and tokenized textual content are concatenated or mixed by easy operations on the very starting of the processing pipeline. This method permits for complete interplay between modalities from the earliest phases of computation, enabling the mannequin to seize delicate correlations and dependencies that is likely to be misplaced in later-stage fusion approaches. 

• Course of: Uncooked modalities -> Characteristic Extraction (low-level options) -> Concatenation/Easy Mixture -> Joint Processing by a single mannequin.
• Professionals: It permits the mannequin to study correlations and interactions between modalities from the earliest phases. It may also be conceptually easier.
• Cons: It may be troublesome to implement successfully if modalities have vastly completely different buildings or scales. The mixed function house can turn out to be very high-dimensional and unwieldy. It forces a “one-size-fits-all” processing method early on, which could not be optimum for every modality.

Instance: Earlier makes an attempt may contain flattening a picture and concatenating it with textual content embeddings earlier than feeding it right into a neural community. That is much less widespread in fashionable refined multimodal LLMs as a result of their limitations.

2. Late Fusion

Late Fusion takes the alternative method, processing every modality independently by specialised networks earlier than combining the outcomes on the determination degree. Right here separate neural networks course of every knowledge sort utilizing architectures optimized for that particular modality like convolutional neural networks for photos, or transformer architectures for textual content and VIT for photos. The outputs from these specialised processors are then mixed utilizing methods akin to weighted averaging, concatenation, or extra refined fusion modules. 

• Course of: Modality A -> Mannequin A -> Output A; Modality B -> Mannequin B -> Output B. Then, Output A and Output B are mixed (utilizing averaging, voting, a small neural community, and so on.).
• Professionals: It permits for optimum, specialised processing of every modality utilizing fashions finest suited to it. It’s easier to implement if you have already got robust unimodal fashions. It’s extra sturdy in lacking modalities.
• Cons: It fails to seize low-level options between modalities as a result of they’re processed in isolation for too lengthy. Additionally, the fusion occurs too late to affect the function studying inside every modality stream.

Instance: A picture classifier identifies objects in a picture, and a textual content classifier analyzes a caption. A separate module then combines/fuses these classifications to say if the caption precisely describes the picture.

3. Mid Fusion

Mid Fusion or intermediate fusion strikes a steadiness between early and late approaches by integrating multimodal data at varied intermediate layers of the community. This technique allows the mannequin to seize each low-level cross-modal interactions and high-level semantic relationships. Mid-fusion architectures usually make use of consideration mechanisms or specialised switch modules that permit data to move between modality-specific processing streams at a number of factors all through the community. The Multimodal Switch Module (MMTM) makes use of this method by utilizing squeeze and excitation operations to recalculate channel-wise options in every CNN stream primarily based on data from a number of modalities.

• Course of: Modality A -> Partial Processing A -> Options A; Modality B -> Partial Processing B -> Options B. Then, Options A and Options B are mixed and fed right into a joint multimodal processing community.
• Professionals: It permits specialised preliminary processing whereas nonetheless enabling the mannequin to study wealthy cross-modal relationships at a deeper function degree. It additionally affords extra flexibility.
• Cons: It may be extra advanced to design and practice. Discovering the optimum level and methodology of fusion might be difficult on this case.

Instance: Most fashionable vision-language fashions (like LLaVA) use this. A picture encoder processes the picture right into a set of function vectors, and a textual content encoder processes the textual content into token embeddings. These are then projected and mixed in a approach that enables a central LLM to take care of each.

Core Encoder Architectures

Let’s now attempt to get an over-the-top understanding of some broadly used encoders which might be utilized within the VLMS.

If you need to study extra about varied Massive Imaginative and prescient Language mannequin architectures click on right here.

CLIP: Contrastive Language-Picture Pre-training

CLIP represents a foundational breakthrough in multimodal studying, introducing a easy but highly effective method to studying joint representations of photos and textual content by contrastive pre-training. The structure consists of two separate encoders: a imaginative and prescient encoder that processes photos and a textual content encoder that processes pure language descriptions. These encoders are educated collectively utilizing a contrastive goal that encourages the mannequin to affiliate photos with their corresponding textual descriptions whereas distinguishing them from unrelated text-image pairs.

The coaching course of for CLIP includes presenting the mannequin with batches (for the sake of understanding the above picture let’s say n=5) of n image-caption pairs, the place every picture is paired with its right textual description. The mannequin computes embeddings for all photos and texts within the batch, creating two units of n-dimensional vectors.

The contrastive loss perform encourages excessive similarity between right image-text pairs whereas penalizing excessive similarity between incorrect pairs. As we are able to visualize within the above picture the diagonal weights can be maximized and the remaining can be penalized. Mathematically, that is expressed as a symmetric cross-entropy loss over similarity scores, the place the temperature parameter controls the sharpness of the distribution. 

CLIP’s effectiveness got here from its potential to study from naturally occurring image-text pairs discovered on the web (400 million scrapped data from the net), eliminating the necessity for manually annotated datasets. This method allows the mannequin to study wealthy semantic relationships that generalize properly to downstream duties. The realized representations show outstanding zero-shot capabilities, permitting the mannequin to carry out picture classification and retrieval duties on classes it has by no means seen throughout coaching. The success of CLIP has impressed quite a few follow-up works and established contrastive pre-training as a dominant methodology in multimodal studying.

Additionally, do think about studying about ViT right here.

SigLIP: Sigmoid Loss for Improved Effectivity

SigLIP represents an evolution of the CLIP structure that addresses among the computational limitations of the unique contrastive method. Whereas CLIP requires computing similarities between all pairs of photos and texts in a batch, SigLIP employs a pairwise sigmoid loss that operates on particular person image-text pairs independently. This modification eliminates the necessity for a worldwide view of all pairwise similarities inside a batch, enabling extra environment friendly scaling to bigger batch sizes whereas sustaining or bettering efficiency.

The sigmoid loss perform utilized in SigLIP affords a number of benefits over the normal contrastive loss. It supplies a extra steady coaching mechanism and higher efficiency with smaller batch sizes, making the method extra accessible with restricted computational sources. The pairwise nature of the loss allows extra versatile coaching configurations and higher dealing with of datasets with various numbers of optimistic examples per pattern.

SigLIP’s structure maintains the dual-encoder construction of CLIP however incorporates architectural enhancements and coaching optimizations that improve each effectivity and effectiveness. The mannequin makes use of separate picture and textual content encoders to generate representations for each modalities, with the sigmoid loss encouraging similarity between matched pairs and dissimilarity between unmatched pairs. This method has demonstrated superior efficiency throughout varied image-text duties whereas providing improved computational effectivity in comparison with conventional contrastive strategies.

RoPE: Rotary Place Embedding

Though RoPE can’t be thought-about as an encoder mannequin, it undoubtedly is an embedding technique broadly utilized in giant language fashions.

Rotary Place Embedding (RoPE) represents a complicated method to encoding positional data in transformer-based architectures. RoPE encodes absolutely the positional data utilizing rotation matrices whereas naturally together with the specific relative place dependencies in self-attention formulations. This method supplies worthwhile properties together with flexibility to increase to any sequence size, decaying inter-token dependency with rising relative distances, and the potential to equip linear self-attention with relative place encoding.

The mathematical basis of RoPE includes making use of rotation matrices to embedding vectors primarily based on their positions within the sequence. This rotation-based method ensures that the dot product between embeddings captures each content material similarity and relative positional relationships. The decay property of RoPE signifies that tokens which might be farther aside within the sequence have naturally lowered consideration weights, which aligns properly with many pure language and multimodal duties the place native context is often extra essential than distant context.

In multimodal functions, RoPE allows fashions to deal with variable-length sequences extra successfully, which is essential when processing multimodal knowledge the place completely different modalities might have completely different temporal or spatial traits. The power to extrapolate to longer sequences than these seen throughout coaching makes RoPE significantly worthwhile for multimodal fashions that must deal with numerous enter codecs and lengths.

Case Research in Imaginative and prescient-Language Fashions

Now, let’s see how these ideas and elements come collectively in some open-sourced influential multimodal LLMs, significantly specializing in how they “see.”

1. LLaVA (Massive Language and Imaginative and prescient Assistant)

LLaVA’s core thought is to show {that a} remarkably easy structure can obtain spectacular visible reasoning capabilities by effectively connecting a pre-trained imaginative and prescient encoder (from CLIP) to a pre-trained Massive Language Mannequin (Vicuna) utilizing a single, trainable linear projection layer. It leverages the robust present capabilities of those unimodal fashions for multimodal understanding.

Coaching Course of

LLaVA makes use of pre-trained Vicuna LLM and CLIP imaginative and prescient encoder elements. The coaching is a 2-stage instruction-tuning process:

Stage 1: Visible Characteristic Alignment (Pre-training)

• Purpose: Train the projection layer to map visible options into the LLM’s phrase embedding house.
• Knowledge: A subset of Conceptual Captions (CC3M), containing image-caption pairs.
• Technique: The picture is fed by the (frozen) CLIP-ViT. The output visible options are handed by the (trainable) linear projection layer. These projected visible tokens are prepended to the tokenized caption. The Vicuna LLM (frozen) is then tasked with autoregressively predicting the caption. Solely the linear projection layer’s weights are up to date.

Stage 2: Instruction Wonderful-tuning (Finish-to-Finish)

• Purpose: Enhance the mannequin’s potential to comply with directions and have interaction in advanced visible dialogue.
• Knowledge: A small, high-quality synthetically generated dataset (LLaVA-Instruct-158K) utilizing GPT-4 to create different questions on photos, detailed descriptions, and complicated reasoning duties. This dataset consists of – Multimodal conversations (58k), Detailed Textual content Descriptions of photos (23k), and Advanced reasoning/advanced visible QA (77k).
• Technique: Each the projection layer and the LLM weights are fine-tuned on this instruction dataset. The enter to the LLM is a mix of projected picture options and a textual instruction/query.

Working

The LLaVA mannequin processes inputs which might be textual content, a picture, or a mix. Right here’s the way it works:

1. Textual content Enter: Vicuna’s native tokenizer and embedding system prepares the supplied textual content (e.g. a query) for the LLM by tokenizing and embedding it.
2. Picture Enter: The CLIP imaginative and prescient encoder (particularly, its Imaginative and prescient Transformer, ViT) extracts wealthy visible options from the picture. These options, sometimes representing picture patches, are a sequence of vectors.
3. Projection: These visible function vectors then move by the MLP Projection Layer. This layer performs a linear transformation, projecting the visible options into the identical dimensionality as Vicuna’s phrase embeddings. This makes the visible data “appear to be” phrase tokens to the LLM.
4. Mixed Enter to LLM: The mannequin then combines the projected visible tokens with the textual content token embeddings (e.g., by prepending the visible tokens to the textual content tokens).
5. LLM Processing (Fusion & Reasoning): This mixed sequence is fed into the Vicuna LLM. The LLM’s consideration mechanisms course of each sorts of tokens concurrently. That is the place “Fusion” occurs, permitting the mannequin to correlate elements of the textual content with related visible tokens. The aim is to realize Joint embedding (a shared illustration house) and Implicit Alignment (connecting visible ideas to textual ones).
6. Output Technology: Based mostly on the processed mixed enter, the LLM autoregressively generates a textual response to the question or instruction.

Simplified Model

LLaVA seems to be at a picture and creates captions for the photographs utilizing CLIP (imaginative and prescient encoder). A particular translator (projection layer) modifications these captions right into a language the Vicuna LLM understands. The Vicuna mind then reads each the translated captions and any precise textual content phrases (like your query). Lastly, the Vicuna mind makes use of all this data to provide you a solution within the textual content.

Encoder-Decoder Structure

Whereas not a standard encoder-decoder within the sequence-to-sequence translation sense, LLaVA makes use of elements that serve these roles:

1. Imaginative and prescient Encoder: A pre-trained CLIP ViT-L/14. This mannequin takes a picture and outputs visible embeddings (options).
2. Language Mannequin (acts as Decoder): Vicuna (an instruction-tuned Llama variant). It takes the visible embeddings (after projection) and textual content embeddings as enter, and autoregressive generates the textual content output.
3. Connector/Projector (The “Bridge”): A single linear MLP layer. That is the important thing new element that interprets visible options from the imaginative and prescient encoder’s house to the LLM’s enter embedding house.

Strengths

• Simplicity & Effectivity: Exceptional efficiency for its comparatively easy structure and environment friendly coaching (particularly Stage 1).
• Leverages Pre-trained Fashions: Successfully makes use of the facility of robust, available pre-trained imaginative and prescient (CLIP) and language (Vicuna) fashions.
• Price-Efficient Wonderful-tuning: The preliminary function alignment stage solely trains a small projection layer, making it computationally cheaper.
• Instruction Following: The LLaVA-Instruct-158K dataset was essential for enabling robust conversational and instruction-following talents.
• Open Supply: Contributed considerably to open analysis in vision-language fashions.

Limitations

• Granularity (Early Variations): Unique LLaVA usually relied on a single international function vector or a small sequence from the picture (e.g., [CLS] token options), which may restrict the understanding of very fine-grained particulars or advanced spatial relationships. (Later variations like LLaVA-1.5 improved this by utilizing extra patch options and an MLP projector).
• Hallucination: Can typically “hallucinate” objects or particulars not current within the picture, a standard problem with LLMs.
• Reasoning Depth: Whereas good, reasoning on very advanced scenes or summary visible ideas is likely to be restricted in comparison with bigger, extra extensively educated fashions.
• Dataset Dependency: Efficiency is closely influenced by the standard and nature of the instruction-tuning dataset.

2. Llama 3 Imaginative and prescient (Llama 3.1 Imaginative and prescient 8B / 70B)

Llama 3 Imaginative and prescient goals to construct state-of-the-art open-source multimodal fashions by integrating a robust imaginative and prescient encoder with the robust base of Llama 3 LLMs. The core thought is to leverage Meta’s developments in LLMs, imaginative and prescient fashions, and large-scale coaching methodologies to create fashions that may carry out advanced visible reasoning, perceive nuanced visible particulars, and comply with intricate directions involving photos and textual content.

Coaching Course of

Llama 3 Imaginative and prescient fashions leverage pre-trained Llama 3 LLMs and highly effective pre-trained imaginative and prescient encoders (e.g., CLIP ViT). The coaching technique sometimes includes:

Stage 1: Massive-Scale Multimodal Pre-training

• Purpose: Train the mannequin elementary visible ideas and their deep alignment with language at a large scale.
• Knowledge: Billions of image-text pairs from numerous sources (e.g., publicly accessible net knowledge, licensed datasets). Meta has entry to huge (anonymized and privacy-preserving) image-text knowledge.
• Technique: The imaginative and prescient encoder, a projector module (e.g., a two-layer MLP), and the Llama 3 LLM are educated collectively. The mannequin learns to foretell textual content related to photos or masked parts of textual content/photos. This stage trains the projector and fine-tunes each the imaginative and prescient encoder and the LLM for multimodal understanding.

Stage 2: Instruction Wonderful-tuning (Finish-to-Finish)

• Purpose: Improve the mannequin’s potential to comply with numerous directions, interact in dialogue, and carry out particular multimodal duties.
• Knowledge: A curated mixture of high-quality multimodal instruction-following datasets. This consists of Visible Query Answering (VQA), picture captioning, visible reasoning, object grounding, Optical Character Recognition (OCR) in photos, chart/diagram understanding, and so on.
• Technique: All the mannequin (or important elements of it) is fine-tuned on these instruction datasets to enhance its helpfulness, security, and task-specific efficiency.
• Scaling: Meta emphasizes scaling legal guidelines, which means Llama 3 Imaginative and prescient advantages from scaling up the LLM measurement (e.g., 8B to 70B), the imaginative and prescient encoder measurement, and the coaching knowledge quantity and high quality.

Working

Llama 3 Imaginative and prescient processes picture and textual content inputs to generate textual outputs.

1. Textual content Enter: Textual content (e.g., questions, directions) is tokenized utilizing Llama 3’s superior tokenizer (e.g., 128k vocabulary) and transformed into token embeddings.
2. Picture Enter: The enter picture is preprocessed (e.g., scaled to a decision like 448×448 for Llama 3.1 Imaginative and prescient). It’s then fed into a robust imaginative and prescient encoder (e.g., a CLIP ViT mannequin). The imaginative and prescient encoder processes the picture and outputs a sequence of visible embeddings, representing quite a few picture patches (e.g., Llama 3.1 Imaginative and prescient produces 144 visible tokens from a CLIP ViT-L/14).
3. Projection: These visible embeddings are handed by a projector module, sometimes a multi-layer perceptron (e.g., a two-layer MLP in Llama 3.1 Imaginative and prescient). The projector transforms these visible options into embeddings which might be appropriate with the Llama 3 LLM’s enter house.
4. Mixed Enter to LLM: The projected visible tokens are mixed with the textual content token embeddings. Particular picture tokens is likely to be used to demarcate visible data throughout the sequence.
5. LLM Processing (Fusion & Reasoning): The Llama 3 LLM processes this interleaved sequence of visible and textual tokens. Its refined consideration mechanisms (Grouped Question Consideration for effectivity with lengthy sequences) permit it to deeply combine and correlate data from each modalities. This permits Joint embedding and Implicit Alignment at a really fine-grained degree.
6. Output Technology: The LLM leverages its huge pre-trained data, detailed visible data, and the textual context to carry out reasoning and generate a coherent and related textual response.

Simplified Model

Llama 3 Imaginative and prescient makes use of a really sharp ViT variant mannequin to have a look at a picture, breaking it down into many detailed image phrases(patch data). A projector makes these detailed picture captions prepared for the super-smart Llama 3 LLM. The Llama 3 mind reads these captions together with any textual content questions you ask it. As a result of the Llama 3 mind is so massive and well-trained, it may perceive advanced issues within the image and provide you with very detailed and clever solutions within the textual content.

Encoder-Decoder Structure

Just like LLaVA, it’s a imaginative and prescient encoder + projector + LLM structure:

1. Imaginative and prescient Encoder: A robust, pre-trained Imaginative and prescient Transformer. For Llama 3.1 Imaginative and prescient, this can be a CLIP ViT mannequin, doubtlessly a big variant.
2. Language Mannequin (acts as a Decoder): The Llama 3 mannequin (e.g., Llama 3 8B or Llama 3 70B), which is an autoregressive decoder.
3. Connector/Projector: A learnable module, sometimes an MLP (e.g., a two-layer MLP for Llama 3.1 Imaginative and prescient) to map the sequences of visible options from the ViT output into the LLM’s enter embedding house.

Strengths

• State-of-the-Artwork Efficiency: Goals for top-tier efficiency on a variety of vision-language benchmarks as a result of scale and superior coaching.
• Scale: Advantages from giant base LLMs (Llama 3 8B, 70B), highly effective imaginative and prescient encoders, and large coaching datasets.
• Sturdy Base LLM: Constructed upon the extremely succesful Llama 3 fashions identified for wonderful textual content era and reasoning.
• Improved Reasoning & Lowered Hallucination: In depth pre-training and fine-tuning on high-quality, numerous knowledge assist enhance reasoning and cut back the tendency to hallucinate.
• Superior Capabilities: Exhibits robust efficiency in areas like OCR, understanding charts/graphs, and fine-grained visible element recognition.
• Architectural Refinements: Leverages LLM developments like Grouped Question Consideration (GQA) for environment friendly dealing with of lengthy sequences (together with visible tokens).

Limitations

• Computational Price: Bigger fashions (like 70B) require important computational sources for coaching and inference.
• Knowledge Dependency & Bias: Efficiency and potential biases are nonetheless depending on the huge datasets used for coaching. Guaranteeing equity and mitigating dangerous biases is an ongoing problem.
• Hallucination: Whereas lowered, the danger of producing believable however incorrect data (hallucination) persists, particularly for out-of-distribution or extremely ambiguous inputs.
• Complexity: The elevated scale and complexity could make debugging, interpretation, and fine-tuning more difficult for end-users in comparison with easier fashions.

Developments in Llama 4

Whereas particular, verified particulars for Llama 4 are nonetheless rising, discussions round its developments usually middle on tackling the inherent challenges of large-scale multimodal studying, significantly by architectural improvements like Combination-of-Specialists (MoE).

1. Addressing Computational Complexity and Scalability with MoE

A key conceptual development for Llama 4 is the efficient implementation of MoE. This structure considerably mitigates computational prices by activating solely a related professional. This enables for enhancing mannequin capability whereas maintaining the computational load for coaching and inference manageable.

Such effectivity is essential for dealing with more and more giant, high-resolution multimodal datasets and lengthy sequence lengths, which might in any other case be bottlenecked by the quadratic scaling of conventional consideration mechanisms. This additionally allows broader scalability options, permitting the mannequin to study from extra in depth and numerous knowledge.

2. Improved Alignment of Heterogeneous Knowledge

With the capability afforded by MoE and developments in coaching methods, Llama 4 would purpose for a extra refined alignment of numerous modalities like photos and textual content. This includes creating extra sturdy representations that may seize modality-specific traits (e.g., spatial correlations in imaginative and prescient, semantic guidelines in textual content) whereas enabling deeper cross-modal understanding and interplay.

Llama4 structure additionally mentions using the Early Fusion mechanism to align the embeddings right into a unified illustration house. Whereas not its main objective, the elevated capability and specialization inside an MoE framework may not directly support in higher dealing with statistical and even temporal discrepancies between modalities if educated on applicable knowledge.

3. Enhanced Robustness and Bias Mitigation

Fashions like Llama 4 are anticipated to include extra superior methods to handle inherited biases and enhance total robustness. Llama 4 would purpose to:

• Implement extra complete bias mitigation methods throughout pre-training and fine-tuning to cut back the amplification of biases by cross-modal interactions.
• Construct better resilience to enter high quality variations, out-of-distribution knowledge, and adversarial assaults that may exploit cross-modal vulnerabilities. The aim is to realize extra dependable and safe efficiency throughout a wider vary of real-world eventualities.

Conclusion

The evolution of multimodal LLMs represents one of the vital important advances in synthetic intelligence, essentially altering how machines understand and work together with the world round us. From the foundational ideas of early and late fusion to the subtle architectures of recent methods like Llama 4, we now have traced the technical journey that has enabled AI methods to grasp and course of a number of modalities with human-like sophistication. The technical foundations we explored together with contrastive studying ideas, joint embedding areas, and alignment mechanisms present the theoretical framework that makes multimodal understanding potential.

Our case research of LLaVA, Llama 3.2 Imaginative and prescient, and Llama 4 illustrate the fast development of multimodal capabilities. LLaVA demonstrated that elegant simplicity may obtain outstanding outcomes by visible instruction tuning. Llama 3.2 Imaginative and prescient confirmed how refined cross-attention mechanisms may allow sturdy multimodal reasoning. Llama 4 represents the present state-of-the-art, introducing mixture-of-experts architectures and unprecedented context lengths that open completely new classes of functions. Within the second a part of this collection, we’ll discover how these Multimodal LLMs are capable of perceive audio.