Python Collection Types and Their Usage

Collection types are a data type in Python programming that stores and organises multiple values within one object. Depending on the use case or situation, the collection type object you would use will change.

Python’s main general-purpose collection types are list, tuple, set, dict, and frozenset.

Comparison

This table below shows how these collection type objects differ. - Ordered - Means that the items in the collection stay in a specific order, which is intrinsic to how to the data in the collection is used and accessed. - Duplicates - Means that the collection allows duplicate values. - Mutable - Means that items can be added, removed, or changed after creation. - Access method - The method by which items are retrieved, which may be with an index number, key, or iteration (i.e. iteratively with a loop). - The first five are built-in Python types, whereas Counter is provided by the standard library’s collections module. - Unpacking means assigning a tuple's items to separate variables at once. For example, here we assign the values x and y to the values stored in the tuple point:

point = (10, 20)
x, y = point

Type	Ordered	Duplicates	Mutable	Access method
list	Yes	Yes	Yes	Numerical index
tuple	Yes	Yes	No	Numerical index or unpacking
set	No	No	Yes	Membership and set operations
frozenset	No	No	No	Membership and set operations
dict	Yes, by insertion	Keys: no Values: yes	Yes	Key
Counter	Yes, by insertion	Stores counts	Yes	Item being counted

Lists: For storing items in order when they may change or repeat

Use a list when: - Order matters (similar to a tuple, dict or Counter) - Duplicates are meaningful (similar to a tuple or Counter) - Values may be added, removed or changed (similar to a set, dict or Counter) - You need access by numerical position (similar to a tuple)

Use a different collection type when: - Items must not change (use a tuple instead) - Values must be unique (use a set instead) - Values need named keys (use a dict instead) - You need to count repeated values (use a Counter instead) - You need an immutable set of unique values (use a frozenset instead)

A simple example for when a list might be appropriate is for storing a patient's successive blood-glucose readings.

This is because: - The items may need to be edited in the case of a clerical error or mixup. - We can have duplicate values from different measurement times; the same reading may likely occur twice depending on the precision of the test. - Readings should remain in measurement order. The order of the readings stays in order unless we decide to change the order.

# Store several glucose readings in a list
glucose_readings = [5.4, 5.7, 5.7, 6.1]

# Display the first reading
print(glucose_readings[0])

# Add a new reading to the end
glucose_readings.append(5.9)

# Display all readings
print(glucose_readings)

A list could also be appropriate for a DNA sequence split into bases. Here, we introduce a T>C mutation in this short sequence through accessing the index value.

sequence = ["A", "T", "G", "C"]
sequence[1] = "C"

print(sequence)
# ['A', 'C', 'G', 'C']

Tuples: For storing related items in order that should not change

A tuple is an ordered but immutable sequence.

Use a tuple when: - Order matters (similar to a list, dict or Counter) - Duplicates are meaningful (similar to a list or Counter) - Values should not change (similar to a frozenset) - You need access by numerical position or unpacking (similar to a list)

Use a different collection type when: - Values may change (use a list instead) - Values must be unique (use a set instead) - Values need named keys (use a dict instead) - You need to count repeated values (use a Counter instead) - You need an unchangeable set of unique values (use a frozenset instead)

A genomic variant can be represented by chromosome, position, reference allele and alternative allele, such as in the code example below. This process below is called unpacking. We assign the tuple’s elements to separate variables. The strict order of values make this practical.

# Store a genomic variant as an unchangeable group of values
variant = ("chr17", 43071077, "A", "G")

# Assign each value to a separate variable
chromosome, position, reference, alternative = variant

# Display the chromosome
print(chromosome)   # chr17

# Display the genomic position
print(position)     # 43071077

A tuple could also be good to store genetic variant data. Consider the tuple below, which has four elements which together represent an A>G mutation at position 43071077 in chromosome 17.

patient1_variant = ("chr17", 43071077, "A", "G")

All four elements of this tuple are required in order to correctly identify it. Not just the values themselves, but also the order of the values are necessary to keep the data organised and protected. This is especially pertinent when your have a set of genetic variants potentially tens of thousands of elements large. Here, patient1 represents one patient, and in this particular case we've decided that we're not doing a quantitative study, and having duplicates of the same variant in unnecessary. This is why we've stored all of the variants as tuples in a set.

patient1_variants_set = {
    ("chr17", 43071077, "A", "G"),
    ("chr7", 117559593, "G", "A"),
}

A dictionary can use a tuple for the same reason as a set. The tuple can provide one stable and complete identifier for a single variant. The difference is that a dictionary can link that identifier to additional information in a key-value pair. all four values are needed to look up the correct information. If the same tuple is used as a key again, Python updates its associated value rather than creating a duplicate key. All four values in the tuple are needed to look up the correct information when it is part of a larger collection. With a dictionary, the same tuple could be used as a key again to update its associated value rather than creating a duplicate key.

For example, here we can make the variant information tuple into the dictionary key, and then connect that key to a string or set of strings containing associated ontologies, diseases, tissue/cell types, etc.

patient1_variants_dict = {
    ("chr17", 43071077, "A", "G"): "pathogenic",
    ("chr7", 117559593, "G", "A"): "uncertain significance",
}

print(patient1_variants_dict[("chr17", 43071077, "A", "G")])
# pathogenic

Sets: store sets of unique items useful for comparisons with other sets

Use a set when: - Every value should be unique (similar to a frozenset) - Order is unimportant (similar to a frozenset) - You need membership tests (i.e. checking if a gene is in a collection of genes: "BRCA1" in genes) (similar to a frozenset or dict keys) - You want to compare groups (i.e. finding which values two groups share or do not share) (similar to a frozenset)

Don’t use a set when: - Duplicate values must be kept or counted (use a list or Counter instead) - Order or numerical position matters. Sets do not support indexing (use a list or tuple instead) - Each value needs associated information (use a dict instead) - The collection must not change after creation (use a frozenset instead)

We can use sets to compare genes containing variants in two patient samples. Because sets are unordered, printing a set may display its items in a different order each time you run the code:

# Store genes observed to have variants in each sample
sample_a_genes = {"BRCA1", "TP53", "CFTR"}
sample_b_genes = {"TP53", "APOE", "CFTR"}

# Find genes shared by both samples
shared_genes = sample_a_genes & sample_b_genes

# Find genes found only in sample A
only_in_a = sample_a_genes - sample_b_genes

# Combine all unique genes from both samples
all_observed_genes = sample_a_genes | sample_b_genes

print(shared_genes)
# {'TP53', 'CFTR'}

print(only_in_a)
# {'BRCA1'}

print(all_observed_genes)
# {'BRCA1', 'TP53', 'CFTR', 'APOE'}

The principal operations for comparing sets are:

Operation	Meaning
a & b	Intersection: values in both
a \| b	Union: values in either or both
a - b	Difference: values in a but not b
a ^ b	Symmetric difference: values in exactly one
value in a	Membership test
a <= b	Test whether a is a subset of b

Frozen sets: fixed groups of unique values

A frozenset has the uniqueness and comparison operations of a set but cannot be modified.

Use a frozenset when: - Every value should be unique (similar to a set) - Order is unimportant (similar to a set) - You need membership tests (similar to a set or dict keys) - You want to compare groups (similar to a set)

Don’t use a frozenset when: - Values may need to be added, removed or changed (use a set instead) - Duplicate values must be kept or counted (use a list or Counter instead) - Order or numerical position matters (use a list or tuple instead) - Each value needs associated information (use a dict instead)

A biomarker is a detectable or measurable biological sign, such as a molecule, genetic change, cell feature, body measurement, or imaging finding that provides information about a normal or abnormal process, condition, or response to treatment.

We can use frozensets for this example analysing patient biomarkers.

• marker_group_a represents a cell that is CD3+ and CD4+: a helper T-cell marker pattern.
• marker_group_b represents a cell that is CD3+ and CD8+: a cytotoxic T-cell marker pattern.

Fixed combinations of biomarkers are used as dictionary keys in the code example below. We've decided to use a frozenset here instead of a set because the biomarkers we use are not going to change. A frozenset cannot be modified, and a set can be modified, which means a frozenset is useable as a dictionary key and a set is not.

Below, we can categorise biomarkers in different groups in the form of frozensets. For the purpose of certain routine tests and experiments, we can assume that a particular cell type has a particular set of unique biomarkersthat are not going to change; perfect for a frozenset. We can compare multiple frozenset values between patients to identify specific cell types.

# Define fixed groups of biomarkers
marker_group_a = frozenset({"CD3", "CD4"})
marker_group_b = frozenset({"CD3", "CD8"})

# Link each marker group to a cell classification
cell_classifications = {
    marker_group_a: "helper T-cell pattern",
    marker_group_b: "cytotoxic T-cell pattern",
}

# Record observed biomarkers; their order does not matter
observed_markers = frozenset({"CD4", "CD3"})

# Look up the matching cell classification
print(cell_classifications[observed_markers])
# helper T-cell pattern

Because sets are unordered, frozenset({"CD3", "CD4"}) and frozenset({"CD4", "CD3"}) are equal. Two frozenset objects are equal when they contain the same unique items, regardless of the order in which those items were written.

An regular set can't be used as a dictionary key because its contents can change after it is created. Dictionary keys must stay the same so Python can reliably find their associated values later. A frozenset cannot be changed, so Python can safely use it as a dictionary key.

A regular set can be changed after its created, and therefore can't be stored inside another set as it can create unresolvable issues. However because a frozenset cannot be changed after it's created, it can be stored safely inside of a set .

# A regular set is created
marker_group = {"CD3", "CD4"}

# Try to put marker_group set inside another group set
try:
    all_marker_groups = {marker_group}
# If not possible, return a TypeError.
except TypeError as error:
    print(error)
# unhashable type: 'set'


# However, a frozenset can be stored inside another set
# Create a frozenset
frozen_marker_group = frozenset({"CD3", "CD4"})

# Put the frozenset inside of a regular set
all_marker_groups = {frozen_marker_group}

# Prints the values with no issues
print(all_marker_groups)
# {frozenset({'CD3', 'CD4'})}

Dictionaries: identifiers mapped to data

A dictionary stores key-value pairs. Use one when each value is associated with a meaningful identifier.

Use a dict when: - Each value needs a meaningful identifier, such as a sample ID (similar to a Counter) - You need to look up a value using its identifier (similar to a Counter) - Keys must be unique, but values may repeat - Key-value pairs may need to be added, removed or changed (similar to a list, set or Counter)

Don’t use a dict when: - You only need unique values, membership tests or group comparisons (use a set or frozenset instead) - Numerical position matters (use a list or tuple instead) - Repeated observations must be kept or counted (use a list or Counter instead) - The collection must not change after creation (use a tuple or frozenset instead)

One example of a use for dictionaries is to map sample identifiers to viral loads. These units could represent a count of viral RNA found in a patient's blood for example. We can map each viral load measurement to a patient sample or patient ID:

# Map sample identifiers to viral RNA counts
viral_loads = {
    "sample_001": 1250,
    "sample_002": 840,
    "sample_003": 2100,
}

# Display the viral load for sample_002
print(viral_loads["sample_002"])
# 840

# Add a viral load measurement for a new sample under the key "sample_004"
viral_loads["sample_004"] = 960

Dictionary values can themselves be collections. Here, we can store many types of sample data under the same sample name as different types of collections depending on the needs of the data.

samples = {
    "sample_001": {
        # Store the unique genes found in this sample
        "genes": {"TP53", "BRCA1"},
        # Store quality scores in a list
        "quality_scores": [38, 37, 39],
    },
    "sample_002": {
        "genes": {"CFTR"},
        "quality_scores": [34, 36, 35],
    },
}

# First select sample_001, then select its genes
print(samples["sample_001"]["genes"])
# {'TP53', 'BRCA1'}  # The order may vary because this is a set

This combines several types:

• The outer dictionary samples maps sample IDs ("sample_001", "sample_002") to records.
• Each of the inner dictionary keys ("genes", "quality_scores") label different data fields.
• A set stored under each sample stores unique genes relating to that sample.
• A list stores an ordered series of quality scores (which could be some kind of experimental data).

Dictionary keys must be hashable; each dictionary key is assigned a 'hash' value, which is an integer calculated from the key value itself. This means that key values cannot be modified after creation. This hash value is the main method Python uses to identify and differentiate between key values in a dictionary, which is what allows it to keep keys unique and find their associated values reliably. Assigning a new value to an existing key will therefore replace the previous value rather than creating a duplicate key. Strings, numbers, tuples, and frozensets can be keys. However, lists and sets can't because they can be modified after they're created. A tuple is only hashable if every value inside it is also hashable, which means a tuple containing a list or set for example still can't be used as a key.

Counter: values and their frequencies

A set records whether a value occurs, but not how many times it occurs. You can use collections.Counter when frequency matters.

Use a Counter when: - You need to count how many times each value occurs - You need to look up a count using the value itself (similar to a dict) - You want to find the most common values - Counts may need to be added, removed or changed (similar to a dict)

Don’t use a Counter when: - Each repeated observation must be a separate element (use a list instead). - Values need associated information other than a count (use a dict instead). - You need only unique values, or need to perform membership tests or group comparisons (use a set or frozenset instead). - Numerical position matters (use a list or tuple instead). - The collection must not change after creation (use a tuple or frozenset instead).

A counter can count repeated items in a collection. Below, we can store responses to a survey in a list, which has duplicates of "yes" or "no" strings. We produce a Counter object called response_counts, which is essentially a dictionary with key:value pairs. However it also comes with some special Counter-specific behaviour and methods (e.g. most_common() for finding the most frequent items).

# Import the Counter collection type
from collections import Counter

# Count survey responses
responses = ["yes", "no", "yes", "yes"]
response_counts = Counter(responses)
# Counter({'yes': 3, 'no': 1})

A counter may also be used on a string, making it useful to count bases in a DNA sequence string.

# Import the Counter collection type
from collections import Counter

# Store a DNA sequence as a string
sequence = "AACCGGGTAA"

# Count how many times each DNA base appears
base_counts = Counter(sequence)

# Display the count for every base
print(base_counts)
# Counter({'A': 4, 'G': 3, 'C': 2, 'T': 1})

# Display the number of G bases
print(base_counts["G"])
# 3