From 9b30428900add41669d066a11e20d256e8d110ef Mon Sep 17 00:00:00 2001 From: Dhruv N Date: Fri, 1 Nov 2024 22:25:51 +0530 Subject: [PATCH] Added Level 3: Fine-Tuning an LLM by Dristro --- Level3/Dristro/README.md | 57 + Level3/Dristro/bert_fine_tune.ipynb | 1467 ++++++++++++++++++++++++++ Level3/Dristro/model_comparision.png | Bin 0 -> 23590 bytes 3 files changed, 1524 insertions(+) create mode 100644 Level3/Dristro/README.md create mode 100644 Level3/Dristro/bert_fine_tune.ipynb create mode 100644 Level3/Dristro/model_comparision.png diff --git a/Level3/Dristro/README.md b/Level3/Dristro/README.md new file mode 100644 index 0000000..98d06d6 --- /dev/null +++ b/Level3/Dristro/README.md @@ -0,0 +1,57 @@ +# Objective +Use the BERT model and the PubMed-RCT dataset to classify a medical abstract. Since the baseline-BERT model is not designed for this purpose, we will be fine-tuning it with the custom dataset in hopes for better performance (foreshadowing - it does perform better). + +--- + +# List of Things +### Model and Its Specs +Offered specs, which will be changed during training: +- **Model**: `bert-base-uncased` + - **Architecture**: BERT Base (12 layers, 768 hidden units, 12 attention heads) + - **Parameters**: Approximately 110 million + - **Casing**: Uncased (i.e., all lowercase inputs) + - **Max Sequence Length**: 512 + - **Total Classes**: 5 (as per the dataset) + +### Hyperparameters +From what I used (feel free to tinker with them): +1. **Batch Size**: 128 +2. **Epochs**: 1 +3. **Max Sequence Length**: 256 (512 was taking too long for training) +4. **Num Workers**: 12 (use as many cores as your CPU allows) +5. **Loss Function**: Adam (`lr=1e-5`, `weight_decay=0.01`) +6. **Trainable Layers**: All layers in BERT +7. **Mixed-Precision Training**: True (I used it for faster training time) + +> *Note: Everything is the same for the untrained model too.* + +--- + +# Results +The trained model performs better than the untrained model. The trained model achieved ≃85% testing accuracy, while the untrained model achieved ≃30% accuracy. The following plot illustrates these results. + +![Trained vs Untrained Model Results](model_comparision.png) + +--- + +# Scope for Improvement +The BERT model was only fine-tuned for 1 epoch with a static learning rate (lr). However, adding a learning rate scheduler and increasing the number of epochs may yield higher accuracy in the model's predictions. + +--- + +# Things to Look Out For When Replicating +1. **Training Speed**: If the model is taking too long to train, try to increase the batch size (if memory allows) and/or reduce the maximum sequence length. If these don't reduce the training time sufficiently, consider try using mixed-precision training, which can improve speed and reduce memory usage. +2. **Dataloader Issue on macOS**: If you're creating the DataLoaders on macOS, set `num_workers = 0` instead of using the number of CPU cores. This is some kinda bug in PyTorch. +3. **Learning Rate**: Since we are fine-tuning the BERT model, try to keep the 'lr' relatively low, this way we can preserve the patterns the BERT model has already learnt. +4. **Evaluation Metrics**: Try to compare the model's using other metrics than the loss and the accuracy. Try to use the model's F1-score to get a deeper insight towards the model's performance. +5. **Device agonistic code**: Make sure to move the model and the dataloader-batches to the correct device. + - **MPS**: for macOS systems `device = "mps" if torch.backends.mps.is_available() else "cpu"` + - **Cuda**: for nvidia GPUs `device = "cuda" if torch.cuda.is_available() else "cpu"` + - **CPU**: for CPU `device = "cpu"` + - **NumPy**: If you would like to use NumPy for any further analysis/operations, then move the model's outputs (logits) to the *CPU*. Same goes for the dataloader-batches. + + +> *Tip: Always save the models, especially if your using Google-colab. Your runtime may get dumped, and you may lose all your progress, so just save the model and download them.* + + +--- \ No newline at end of file diff --git a/Level3/Dristro/bert_fine_tune.ipynb b/Level3/Dristro/bert_fine_tune.ipynb new file mode 100644 index 0000000..e4a34f3 --- /dev/null +++ b/Level3/Dristro/bert_fine_tune.ipynb @@ -0,0 +1,1467 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "9dfc6338-9326-4af7-919d-ef5f9db86faa", + "metadata": { + "id": "9dfc6338-9326-4af7-919d-ef5f9db86faa" + }, + "source": [ + "# Getting the data ready" + ] + }, + { + "cell_type": "markdown", + "id": "26227c33-be5f-4acc-b8a0-b5c656753042", + "metadata": { + "id": "26227c33-be5f-4acc-b8a0-b5c656753042" + }, + "source": [ + "## Parse the text" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "9d00b01d-fcac-43dc-ab75-8a2131393e0d", + "metadata": { + "id": "9d00b01d-fcac-43dc-ab75-8a2131393e0d" + }, + "outputs": [], + "source": [ + "import re\n", + "from torch.utils.data import Dataset, DataLoader\n", + "from transformers import BertTokenizer\n", + "\n", + "class MedicalAbstractDataset(Dataset):\n", + " def __init__(self, file_path, tokenizer, label_map, max_length=512):\n", + " self.examples = []\n", + " self.tokenizer = tokenizer\n", + " self.label_map = label_map\n", + " self.max_length = max_length\n", + "\n", + " with open(file_path, 'r') as file:\n", + " current_id = None\n", + " for line in file:\n", + " line = line.strip()\n", + " if line.startswith(\"###\"): # New document ID\n", + " current_id = line\n", + " elif re.match(r'^[A-Z]+', line.split(\"\\t\")[0]):\n", + " # Split line to get label and text\n", + " section = line.split(\"\\t\")\n", + " label = section[0]\n", + " text = section[1]\n", + "\n", + " if label in self.label_map:\n", + " self.examples.append((current_id, label, text))\n", + "\n", + " def __len__(self):\n", + " return len(self.examples)\n", + "\n", + " def __getitem__(self, idx):\n", + " _, label, text = self.examples[idx]\n", + " label_id = self.label_map[label]\n", + " encoding = self.tokenizer(\n", + " text,\n", + " padding='max_length',\n", + " truncation=True,\n", + " max_length=self.max_length,\n", + " return_tensors='pt'\n", + " )\n", + "\n", + " input_ids = encoding['input_ids'].squeeze()\n", + " attention_mask = encoding['attention_mask'].squeeze()\n", + "\n", + " return {\n", + " 'input_ids': input_ids,\n", + " 'attention_mask': attention_mask,\n", + " 'label': torch.tensor(label_id, dtype=torch.long)\n", + " }" + ] + }, + { + "cell_type": "markdown", + "id": "2043aa20-e1fa-457b-939c-f8c484bd39a9", + "metadata": { + "id": "2043aa20-e1fa-457b-939c-f8c484bd39a9" + }, + "source": [ + "## Create output label mappings (classes)" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "9c1b0795-5e58-42d3-b737-de173f5c452e", + "metadata": { + "id": "9c1b0795-5e58-42d3-b737-de173f5c452e" + }, + "outputs": [], + "source": [ + "label_map = {\n", + " 'BACKGROUND': 0,\n", + " 'OBJECTIVE': 1,\n", + " 'METHODS': 2,\n", + " 'RESULTS': 3,\n", + " 'CONCLUSIONS': 4\n", + "}" + ] + }, + { + "cell_type": "markdown", + "id": "6e4659ba-c1e5-4a2a-9a8e-37810676b223", + "metadata": { + "id": "6e4659ba-c1e5-4a2a-9a8e-37810676b223" + }, + "source": [ + "## Build BERT tokenizer" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "6360387d-5fb2-4d22-ac16-8389a292eb4b", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "6360387d-5fb2-4d22-ac16-8389a292eb4b", + "outputId": "3e63806e-11c2-4236-e84d-894d3c704999" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:89: UserWarning: \n", + "The secret `HF_TOKEN` does not exist in your Colab secrets.\n", + "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n", + "You will be able to reuse this secret in all of your notebooks.\n", + "Please note that authentication is recommended but still optional to access public models or datasets.\n", + " warnings.warn(\n", + "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884\n", + " warnings.warn(\n" + ] + } + ], + "source": [ + "tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')" + ] + }, + { + "cell_type": "markdown", + "id": "dbfbfe04-4712-429c-af1d-6521f6093d4d", + "metadata": { + "id": "dbfbfe04-4712-429c-af1d-6521f6093d4d" + }, + "source": [ + "## Create torch dataloaders" + ] + }, + { + "cell_type": "code", + "source": [ + "!wget \"https://raw.githubusercontent.com/Franck-Dernoncourt/pubmed-rct/refs/heads/master/PubMed_20k_RCT/train.txt\"\n", + "!wget \"https://raw.githubusercontent.com/Franck-Dernoncourt/pubmed-rct/refs/heads/master/PubMed_20k_RCT/test.txt\"" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "I7aEOf1lM5VH", + "outputId": "9b8a73b9-5533-47a8-c194-476909a3596e" + }, + "id": "I7aEOf1lM5VH", + "execution_count": 4, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "--2024-11-01 15:23:30-- https://raw.githubusercontent.com/Franck-Dernoncourt/pubmed-rct/refs/heads/master/PubMed_20k_RCT/train.txt\n", + "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n", + "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n", + "HTTP request sent, awaiting response... 200 OK\n", + "Length: 29384101 (28M) [text/plain]\n", + "Saving to: ‘train.txt.2’\n", + "\n", + "\rtrain.txt.2 0%[ ] 0 --.-KB/s \rtrain.txt.2 100%[===================>] 28.02M --.-KB/s in 0.07s \n", + "\n", + "2024-11-01 15:23:30 (416 MB/s) - ‘train.txt.2’ saved [29384101/29384101]\n", + "\n", + "--2024-11-01 15:23:30-- https://raw.githubusercontent.com/Franck-Dernoncourt/pubmed-rct/refs/heads/master/PubMed_20k_RCT/test.txt\n", + "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.110.133, ...\n", + "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.\n", + "HTTP request sent, awaiting response... 200 OK\n", + "Length: 4889845 (4.7M) [text/plain]\n", + "Saving to: ‘test.txt.2’\n", + "\n", + "test.txt.2 100%[===================>] 4.66M --.-KB/s in 0.01s \n", + "\n", + "2024-11-01 15:23:30 (381 MB/s) - ‘test.txt.2’ saved [4889845/4889845]\n", + "\n" + ] + } + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "290bae08-5fba-4c4c-8980-71e70394783c", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "290bae08-5fba-4c4c-8980-71e70394783c", + "outputId": "55356e32-5ce4-4693-be9a-284fd7bd0791" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(,\n", + " )" + ] + }, + "metadata": {}, + "execution_count": 5 + } + ], + "source": [ + "import torch\n", + "import os\n", + "root_dir = os.getcwd()\n", + "train_dir = root_dir + \"/train.txt\"\n", + "test_dir = root_dir + \"/test.txt\"\n", + "cpu_count = os.cpu_count()\n", + "BATCH_SIZE = 128\n", + "NUM_WORKERS = cpu_count\n", + "MAX_LENGTH = 256\n", + "\n", + "train_data = MedicalAbstractDataset(train_dir, tokenizer, label_map, max_length=MAX_LENGTH)\n", + "test_data = MedicalAbstractDataset(test_dir, tokenizer, label_map, max_length=MAX_LENGTH)\n", + "\n", + "train_dataloader = DataLoader(train_data, batch_size=BATCH_SIZE, shuffle=True, num_workers=NUM_WORKERS)\n", + "test_dataloader = DataLoader(test_data, batch_size=BATCH_SIZE, shuffle=True, num_workers=NUM_WORKERS)\n", + "\n", + "train_dataloader, test_dataloader" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "3c5abea4-a119-4ff3-a965-c693bd9aaef4", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "3c5abea4-a119-4ff3-a965-c693bd9aaef4", + "outputId": "e7878e34-ac95-4e61-bf3f-a0ff5090d72c" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Input IDs shape: torch.Size([128, 256])\n", + "Attention mask shape: torch.Size([128, 256])\n", + "Labels shape: torch.Size([128])\n" + ] + } + ], + "source": [ + "### Verify the train dataloader\n", + "for batch in train_dataloader:\n", + " input_ids = batch['input_ids']\n", + " attention_mask = batch['attention_mask']\n", + " labels = batch['label']\n", + "\n", + " print(\"Input IDs shape:\", input_ids.shape)\n", + " print(\"Attention mask shape:\", attention_mask.shape)\n", + " print(\"Labels shape:\", labels.shape)\n", + "\n", + " break" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "00f2a14b-7824-47f2-8b73-463990d41e84", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "00f2a14b-7824-47f2-8b73-463990d41e84", + "outputId": "15216650-9cd4-4c21-b683-9f65558c5f85" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Input IDs shape: torch.Size([128, 256])\n", + "Attention mask shape: torch.Size([128, 256])\n", + "Labels shape: torch.Size([128])\n" + ] + } + ], + "source": [ + "### Verify the test dataloader\n", + "for batch in test_dataloader:\n", + " input_ids = batch['input_ids']\n", + " attention_mask = batch['attention_mask']\n", + " labels = batch['label']\n", + "\n", + " print(\"Input IDs shape:\", input_ids.shape)\n", + " print(\"Attention mask shape:\", attention_mask.shape)\n", + " print(\"Labels shape:\", labels.shape)\n", + "\n", + " break" + ] + }, + { + "cell_type": "markdown", + "id": "322d11d0-b39d-4a3e-8c61-a68cb54b7001", + "metadata": { + "id": "322d11d0-b39d-4a3e-8c61-a68cb54b7001" + }, + "source": [ + "# Modelling" + ] + }, + { + "cell_type": "markdown", + "id": "57d94166-b774-4d65-b4ae-abc7c7d9df0e", + "metadata": { + "id": "57d94166-b774-4d65-b4ae-abc7c7d9df0e" + }, + "source": [ + "## Getting the model ready" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "1ec0fb60-37e5-4995-af0c-ae82fea69afc", + "metadata": { + "scrolled": true, + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "1ec0fb60-37e5-4995-af0c-ae82fea69afc", + "outputId": "4cdc5579-fb5c-4805-cec4-fa94d116175e" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.bias', 'classifier.weight']\n", + "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n" + ] + }, + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "BertForSequenceClassification(\n", + " (bert): BertModel(\n", + " (embeddings): BertEmbeddings(\n", + " (word_embeddings): Embedding(30522, 768, padding_idx=0)\n", + " (position_embeddings): Embedding(512, 768)\n", + " (token_type_embeddings): Embedding(2, 768)\n", + " (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " )\n", + " (encoder): BertEncoder(\n", + " (layer): ModuleList(\n", + " (0-11): 12 x BertLayer(\n", + " (attention): BertAttention(\n", + " (self): BertSdpaSelfAttention(\n", + " (query): Linear(in_features=768, out_features=768, bias=True)\n", + " (key): Linear(in_features=768, out_features=768, bias=True)\n", + " (value): Linear(in_features=768, out_features=768, bias=True)\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " )\n", + " (output): BertSelfOutput(\n", + " (dense): Linear(in_features=768, out_features=768, bias=True)\n", + " (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " )\n", + " )\n", + " (intermediate): BertIntermediate(\n", + " (dense): Linear(in_features=768, out_features=3072, bias=True)\n", + " (intermediate_act_fn): GELUActivation()\n", + " )\n", + " (output): BertOutput(\n", + " (dense): Linear(in_features=3072, out_features=768, bias=True)\n", + " (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " )\n", + " )\n", + " )\n", + " )\n", + " (pooler): BertPooler(\n", + " (dense): Linear(in_features=768, out_features=768, bias=True)\n", + " (activation): Tanh()\n", + " )\n", + " )\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " (classifier): Linear(in_features=768, out_features=5, bias=True)\n", + ")" + ] + }, + "metadata": {}, + "execution_count": 8 + } + ], + "source": [ + "from transformers import BertForSequenceClassification\n", + "num_classes = len(label_map) # 5\n", + "bert_model = BertForSequenceClassification.from_pretrained(\"bert-base-uncased\", num_labels=num_classes)\n", + "\n", + "device = \"cuda\" if torch.cuda.is_available() else \"cpu\" # | Run this for nvidia GPU\n", + "#device = \"mps\" if torch.backends.mps.is_available() else \"cpu\" # | Run this for mac\n", + "device = torch.device(device)\n", + "bert_model.to(device)\n", + "\n", + "bert_model" + ] + }, + { + "cell_type": "markdown", + "id": "ffba4ff4-36b4-456c-9a5f-5dea33ddb460", + "metadata": { + "id": "ffba4ff4-36b4-456c-9a5f-5dea33ddb460" + }, + "source": [ + "## Training" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "a6165067-13fc-4073-a55b-2708ee904363", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "a6165067-13fc-4073-a55b-2708ee904363", + "outputId": "e89ea448-7491-46f0-d6c9-0a73ad78681c" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(CrossEntropyLoss(),\n", + " Adam (\n", + " Parameter Group 0\n", + " amsgrad: False\n", + " betas: (0.9, 0.999)\n", + " capturable: False\n", + " differentiable: False\n", + " eps: 1e-08\n", + " foreach: None\n", + " fused: None\n", + " lr: 1e-05\n", + " maximize: False\n", + " weight_decay: 0.01\n", + " ))" + ] + }, + "metadata": {}, + "execution_count": 9 + } + ], + "source": [ + "from torch import nn\n", + "loss_fn = nn.CrossEntropyLoss()\n", + "optimizer = torch.optim.Adam(bert_model.parameters(), lr=1e-5, weight_decay=0.01)\n", + "loss_fn, optimizer" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "003ddd59-8d6d-4aa4-a743-5af374ca31ae", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "003ddd59-8d6d-4aa4-a743-5af374ca31ae", + "outputId": "938df0c3-a4ed-4c79-8941-2bdfec462bc7" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "device(type='cuda')" + ] + }, + "metadata": {}, + "execution_count": 10 + } + ], + "source": [ + "device" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "bb4fa7b6-d6ec-454c-8e91-3c3c56d5764e", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "bb4fa7b6-d6ec-454c-8e91-3c3c56d5764e", + "outputId": "87f48248-b5f3-4a4d-c336-1841ed863538" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + ":3: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.\n", + " scaler = GradScaler()\n" + ] + } + ], + "source": [ + "from torch.cuda.amp import GradScaler, autocast\n", + "from tqdm.auto import tqdm\n", + "scaler = GradScaler()\n", + "def train_bert(model,\n", + " train_dataloader,\n", + " test_dataloader,\n", + " loss_fn,\n", + " optimizer,\n", + " epochs,\n", + " device,\n", + " print_freq=10):\n", + " results = {\n", + " \"train_loss\": [],\n", + " \"train_accuracy\": [],\n", + " \"test_loss\": [],\n", + " \"test_accuracy\": []\n", + " }\n", + "\n", + " for epoch in range(epochs):\n", + " model.train()\n", + " train_loss = 0\n", + " train_correct = 0\n", + " train_samples = 0\n", + "\n", + " for step, batch in enumerate(tqdm(train_dataloader, desc=f\"Epoch {epoch + 1}/{epochs}\", leave=False)):\n", + " input_ids = batch['input_ids'].to(device)\n", + " attention_mask = batch['attention_mask'].to(device)\n", + " labels = batch['label'].to(device)\n", + "\n", + " with autocast(): # Enable mixed precision\n", + " outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)\n", + " loss = outputs.loss\n", + " logits = outputs.logits\n", + "\n", + " optimizer.zero_grad()\n", + " scaler.scale(loss).backward() # Scale the loss\n", + " scaler.step(optimizer)\n", + " scaler.update()\n", + "\n", + " train_loss += loss.item()\n", + " predictions = torch.argmax(logits, dim=1)\n", + " train_correct += (predictions == labels).sum().item()\n", + " train_samples += labels.size(0)\n", + "\n", + " if (step + 1) % print_freq == 0:\n", + " avg_loss = train_loss / (step + 1)\n", + " accuracy = train_correct / train_samples\n", + " print(f\"Step [{step + 1}/{len(train_dataloader)}], Loss: {avg_loss:.4f}, Accuracy: {accuracy:.4f}\")\n", + "\n", + " avg_train_loss = train_loss / len(train_dataloader)\n", + " train_accuracy = train_correct / train_samples\n", + " results[\"train_loss\"].append(avg_train_loss)\n", + " results[\"train_accuracy\"].append(train_accuracy)\n", + "\n", + " # Evaluation phase (similar adjustments for mixed precision)\n", + " model.eval()\n", + " test_loss = 0\n", + " test_correct = 0\n", + " test_samples = 0\n", + " with torch.no_grad():\n", + " for batch in test_dataloader:\n", + " input_ids = batch['input_ids'].to(device)\n", + " attention_mask = batch['attention_mask'].to(device)\n", + " labels = batch['label'].to(device)\n", + "\n", + " with autocast():\n", + " outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)\n", + " loss = outputs.loss\n", + " logits = outputs.logits\n", + "\n", + " test_loss += loss.item()\n", + " preds = torch.argmax(logits, dim=1)\n", + " test_correct += (preds == labels).sum().item()\n", + " test_samples += labels.size(0)\n", + "\n", + " avg_test_loss = test_loss / len(test_dataloader)\n", + " test_accuracy = test_correct / test_samples\n", + " results[\"test_loss\"].append(avg_test_loss)\n", + " results[\"test_accuracy\"].append(test_accuracy)\n", + "\n", + " print(f\"Epoch [{epoch + 1}/{epochs}] - Train Loss: {avg_train_loss:.4f}, Train Accuracy: {train_accuracy:.4f}, \"\n", + " f\"Test Loss: {avg_test_loss:.4f}, Test Accuracy: {test_accuracy:.4f}\")\n", + " return results\n" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "10d8cb8d-ae31-4833-962e-22314c298b9a", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 394, + "referenced_widgets": [ + "f559502a61384d8895066de4905d7bdf", + "af39de1ed03a47b7ad9513ef8d332a8c", + "24d8020746bb464c8cd69bc35fa4cadf", + "cfee43bbf82049efb6fbbe0490930f31", + "00f422a7380b488a80c69b84d8479ae4", + "498135911ebc4b05945003fc0396e2b3", + "14c23facacb440638f9ef71480066f2c", + "a0a9629453294ac6a6da8442ced3aa91", + "ba67a3d57664494d844ee60eb2951d54", + "57f1b16a8e804b24b7710c4e47a32ba3", + "2788611a893f45819c9761f6d20108b8" + ] + }, + "id": "10d8cb8d-ae31-4833-962e-22314c298b9a", + "outputId": "37c27f9e-4186-4ba2-aa6e-176c43fc390f" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "Epoch 1/1: 0%| | 0/1407 [00:00:30: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.\n", + " with autocast(): # Enable mixed precision\n", + "/usr/local/lib/python3.10/dist-packages/torch/autograd/graph.py:825: UserWarning: cuDNN SDPA backward got grad_output.strides() != output.strides(), attempting to materialize a grad_output with matching strides... (Triggered internally at ../aten/src/ATen/native/cudnn/MHA.cpp:674.)\n", + " return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass\n" + ] + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Step [100/1407], Loss: 1.0833, Accuracy: 0.6156\n", + "Step [200/1407], Loss: 0.8648, Accuracy: 0.6968\n", + "Step [300/1407], Loss: 0.7599, Accuracy: 0.7359\n", + "Step [400/1407], Loss: 0.6936, Accuracy: 0.7589\n", + "Step [500/1407], Loss: 0.6493, Accuracy: 0.7743\n", + "Step [600/1407], Loss: 0.6154, Accuracy: 0.7862\n", + "Step [700/1407], Loss: 0.5911, Accuracy: 0.7944\n", + "Step [800/1407], Loss: 0.5728, Accuracy: 0.8005\n", + "Step [900/1407], Loss: 0.5575, Accuracy: 0.8053\n", + "Step [1000/1407], Loss: 0.5446, Accuracy: 0.8095\n", + "Step [1100/1407], Loss: 0.5334, Accuracy: 0.8131\n", + "Step [1200/1407], Loss: 0.5242, Accuracy: 0.8160\n", + "Step [1300/1407], Loss: 0.5156, Accuracy: 0.8187\n", + "Step [1400/1407], Loss: 0.5082, Accuracy: 0.8209\n" + ] + }, + { + "output_type": "stream", + "name": "stderr", + "text": [ + ":66: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.\n", + " with autocast():\n" + ] + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Epoch [1/1] - Train Loss: 0.5076, Train Accuracy: 0.8211, Test Loss: 0.4177, Test Accuracy: 0.8490\n" + ] + } + ], + "source": [ + "### Training the model\n", + "EPOCHS = 1\n", + "\n", + "results = train_bert(\n", + " model = bert_model,\n", + " train_dataloader = train_dataloader,\n", + " test_dataloader = test_dataloader,\n", + " loss_fn = loss_fn,\n", + " optimizer = optimizer,\n", + " epochs = EPOCHS,\n", + " device = device,\n", + " print_freq = 100\n", + ")" + ] + }, + { + "cell_type": "code", + "source": [ + "### Save the model\n", + "save_path = root_dir + \"/bert_fine_tune_1.pth\"\n", + "torch.save(bert_model.state_dict(), save_path)\n", + "print(f\"Model state dictionary saved to {save_path}\")" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "vrzFt0iJUeI_", + "outputId": "3c3d1041-6dfa-4004-937d-ff5bf5598a94" + }, + "id": "vrzFt0iJUeI_", + "execution_count": 16, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Model state dictionary saved to /content/bert_fine_tune_1.pth\n" + ] + } + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "b1284643-46a1-432b-9ad9-3ebfad4b0bef", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "b1284643-46a1-432b-9ad9-3ebfad4b0bef", + "outputId": "64926fa0-ca7b-4a7c-bfa6-e4a1dc8218a8" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Predicted label: OBJECTIVE\n" + ] + } + ], + "source": [ + "itos = {\n", + " 0: 'BACKGROUND',\n", + " 1: 'OBJECTIVE',\n", + " 2: 'METHODS',\n", + " 3: 'RESULTS',\n", + " 4: 'CONCLUSIONS'\n", + "}\n", + "bert_model.to(device)\n", + "\n", + "def predict_sample(text, model, tokenizer, label_map):\n", + " # Preprocess the input text\n", + " encoding = tokenizer(\n", + " text,\n", + " padding='max_length',\n", + " truncation=True,\n", + " max_length=512,\n", + " return_tensors='pt'\n", + " )\n", + "\n", + " # Move tensors to the appropriate device\n", + " input_ids = encoding['input_ids'].to(device)\n", + " attention_mask = encoding['attention_mask'].to(device)\n", + "\n", + " # Forward pass (no gradients needed for inference)\n", + " with torch.no_grad():\n", + " outputs = model(input_ids=input_ids, attention_mask=attention_mask)\n", + " logits = outputs.logits\n", + "\n", + " # Get the predicted label\n", + " predicted_class = torch.argmax(logits, dim=1).item()\n", + " predicted_label = itos[predicted_class]\n", + "\n", + " return predicted_label\n", + "\n", + "# Sample input text\n", + "sample_text = \"This study analyzed liver function abnormalities in heart failure patients admitted with severe acute decompensated heart failure.\"\n", + "predicted_label = predict_sample(sample_text, bert_model, tokenizer, label_map)\n", + "print(f\"Predicted label: {predicted_label}\")" + ] + }, + { + "cell_type": "code", + "source": [ + "results" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Fqq4TTRFYG3h", + "outputId": "30399d56-232e-4a25-e057-553afce8e619" + }, + "id": "Fqq4TTRFYG3h", + "execution_count": 31, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "{'train_loss': [0.507557946100418],\n", + " 'train_accuracy': [0.8211064207953788],\n", + " 'test_loss': [0.41773838335174623],\n", + " 'test_accuracy': [0.8490459598473535]}" + ] + }, + "metadata": {}, + "execution_count": 31 + } + ] + }, + { + "cell_type": "code", + "source": [ + "import matplotlib.pyplot as plt\n", + "\n", + "train_loss = results['train_loss'][0]\n", + "train_accuracy = results['train_accuracy'][0]\n", + "test_loss = results['test_loss'][0]\n", + "test_accuracy = results['test_accuracy'][0]\n", + "\n", + "# Data\n", + "loss_metrics = ['Train Loss', 'Test Loss']\n", + "loss_values = [train_loss, test_loss]\n", + "accuracy_metrics = ['Train Accuracy', 'Test Accuracy']\n", + "accuracy_values = [train_accuracy, test_accuracy]\n", + "\n", + "# Loss\n", + "plt.figure(figsize=(10, 5))\n", + "plt.subplot(1, 2, 1)\n", + "plt.bar(loss_metrics, loss_values, color=['blue', 'orange'])\n", + "plt.ylabel('Loss')\n", + "plt.title('Training and Testing Loss')\n", + "\n", + "# Accuracy\n", + "plt.subplot(1, 2, 2)\n", + "plt.bar(accuracy_metrics, accuracy_values, color=['blue', 'orange'])\n", + "plt.ylim(0, 1)\n", + "plt.ylabel('Accuracy')\n", + "plt.title('Training and Testing Accuracy')\n", + "\n", + "plt.tight_layout()\n", + "plt.show()" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 507 + }, + "id": "ubCnJH-AXJ5v", + "outputId": "ab52c2fa-017a-4f3e-d4b8-839b7425d35e" + }, + "id": "ubCnJH-AXJ5v", + "execution_count": 39, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA90AAAHqCAYAAAAZLi26AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABUmUlEQVR4nO3deVhV5f7+8XsDskEUHFAGRVCcJ3AkM4cKo/SYWuZQJ5BMy6mBOpWZ4pBimubJTMv5mNNxzMosxfyaZVkqpTmU8whOCYoeMFi/P/q5z9kxiiw2yPt1Xfu6XM9+1tqftWTxcO81WQzDMAQAAAAAAAqdk6MLAAAAAADgTkXoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugG/qJfv34KCgoq0LyjR4+WxWIp3IKKmWPHjslisWjBggWOLiVPCxYskMVi0bFjxxxdCgBAjLF5YYwF7kyEbpQYFoslX68tW7Y4utRSLygoKF//V4X1R8WECRO0du3aQllWYbn5x+GFCxccXQoA5IkxtuRgjLX3yiuvyGKxqHfv3o4uBciRxTAMw9FFAPnx0Ucf2U3/61//0saNG7Vo0SK79k6dOsnHx6fAn3Pjxg1lZmbKarXe8rx//PGH/vjjD7m5uRX484u7Y8eOqWbNmpo/f7769euXbZ+1a9fq6tWrtun169dr6dKleuedd+Tt7W1rv/vuu1WrVq3brqlcuXLq2bNnlj8wMjIydOPGDVmt1iI/OjJ69GiNGTNG58+ft1tnACiOGGOLB8bYW2MYhmrUqCEXFxclJSUpKSlJ5cuXd0gtQG5cHF0AkF9///vf7aa/++47bdy4MUv7X127dk1ly5bN9+eUKVOmQPVJkouLi1xc2K26d+9uN52YmKilS5eqe/fuBT6tsCCcnZ3l7OxcZJ8HACUVY2zJwRj7X1u2bNGpU6e0efNmRUREaPXq1YqKinJoTTm51X0FdxZOL8cdpWPHjmrcuLF27typ9u3bq2zZsnr99dclSR9//LG6dOkif39/Wa1WBQcHa9y4ccrIyLBbxl+vN7t5fdXbb7+tDz/8UMHBwbJarWrVqpV++OEHu3mzu97MYrFo6NChWrt2rRo3biyr1apGjRppw4YNWerfsmWLWrZsKTc3NwUHB+uDDz7I9zVsX3/9tR577DHVqFFDVqtVAQEBevHFF3X9+vUs61euXDmdPn1a3bt3V7ly5VSlShW9/PLLWbbF5cuX1a9fP3l5ealChQqKiorS5cuX86wlvz766CO1aNFC7u7uqlSpkvr06aOTJ0/a9fntt9/06KOPytfXV25ubqpevbr69Omj5ORkSX9u39TUVC1cuNB2St3NowPZXW8WFBSkv/3tb9q2bZtat24tNzc31apVS//617+y1Pfzzz+rQ4cOcnd3V/Xq1fXmm29q/vz5hXoN2+bNm9WuXTt5eHioQoUK6tatm/bv32/X58qVK3rhhRcUFBQkq9WqqlWrqlOnTtq1a1e+txMA3C7GWMbY4jbGLl68WA0bNtS9996r8PBwLV68ONt+p0+fVv/+/W0/nzVr1tSgQYOUnp5u63P58mW9+OKLtrG2evXqioyMtF0mltM17Fu2bMly6UVh7CuS9P3336tz586qWLGiPDw81LRpU/3zn/+UJNu22r17d5b5JkyYIGdnZ50+fTpf2xHm4+tC3HEuXryohx56SH369NHf//5322lwCxYsULly5RQTE6Ny5cpp8+bNGjVqlFJSUjR58uQ8l7tkyRJduXJFzzzzjCwWiyZNmqRHHnlER44cyfOb+23btmn16tUaPHiwypcvr3fffVePPvqoTpw4ocqVK0uSdu/erQcffFB+fn4aM2aMMjIyNHbsWFWpUiVf671ixQpdu3ZNgwYNUuXKlbVjxw5Nnz5dp06d0ooVK+z6ZmRkKCIiQmFhYXr77be1adMmTZkyRcHBwRo0aJCkP0/Z6tatm7Zt26Znn31WDRo00Jo1awrtG+Tx48dr5MiR6tWrl55++mmdP39e06dPV/v27bV7925VqFBB6enpioiIUFpamoYNGyZfX1+dPn1an376qS5fviwvLy8tWrRITz/9tFq3bq2BAwdKkoKDg3P97EOHDqlnz57q37+/oqKiNG/ePPXr108tWrRQo0aNJP05QN97772yWCwaPny4PDw8NGfOnAKdEpmTTZs26aGHHlKtWrU0evRoXb9+XdOnT1fbtm21a9cu2x+mzz77rFauXKmhQ4eqYcOGunjxorZt26b9+/erefPm+dpOAFAYGGMZY4vLGJuWlqZVq1bppZdekiT17dtX0dHRSkxMlK+vr63fmTNn1Lp1a12+fFkDBw5U/fr1dfr0aa1cuVLXrl2Tq6urrl69qnbt2mn//v166qmn1Lx5c124cEHr1q3TqVOnCnSZ2O3uKxs3btTf/vY3+fn56fnnn5evr6/279+vTz/9VM8//7x69uypIUOGaPHixWrWrJndZy9evFgdO3ZUtWrVbrlumMQASqghQ4YYf/0R7tChgyHJmDVrVpb+165dy9L2zDPPGGXLljX+85//2NqioqKMwMBA2/TRo0cNSUblypWNS5cu2do//vhjQ5LxySef2NpiY2Oz1CTJcHV1NQ4dOmRr++mnnwxJxvTp021tXbt2NcqWLWucPn3a1vbbb78ZLi4uWZaZnezWLy4uzrBYLMbx48ft1k+SMXbsWLu+zZo1M1q0aGGbXrt2rSHJmDRpkq3tjz/+MNq1a2dIMubPn59nTTdNnjzZkGQcPXrUMAzDOHbsmOHs7GyMHz/ert+ePXsMFxcXW/vu3bsNScaKFStyXb6Hh4cRFRWVpX3+/Pl2n2sYhhEYGGhIMrZu3WprO3funGG1Wo2XXnrJ1jZs2DDDYrEYu3fvtrVdvHjRqFSpUpZlZufmz8L58+dz7BMaGmpUrVrVuHjxoq3tp59+MpycnIzIyEhbm5eXlzFkyJAcl5Pf7QQA+cUYm/f6McY6bow1DMNYuXKlIcn47bffDMMwjJSUFMPNzc1455137PpFRkYaTk5Oxg8//JBlGZmZmYZhGMaoUaMMScbq1atz7JPd+hqGYXz11VeGJOOrr76ytd3uvvLHH38YNWvWNAIDA43ff/8923oMwzD69u1r+Pv7GxkZGba2Xbt23fLPEMzH6eW441itVkVHR2dpd3d3t/37ypUrunDhgtq1a6dr167pwIEDeS63d+/eqlixom26Xbt2kqQjR47kOW94eLjdN8NNmzaVp6enbd6MjAxt2rRJ3bt3l7+/v61f7dq19dBDD+W5fMl+/VJTU3XhwgXdfffdMgwj21OPnn32Wbvpdu3a2a3L+vXr5eLiYvtWXvrz+q1hw4blq57crF69WpmZmerVq5cuXLhge/n6+qpOnTr66quvJMl2hPaLL77QtWvXbvtzb2rYsKHt/0+SqlSponr16tmt/4YNG9SmTRuFhoba2ipVqqQnnniiUGo4e/asEhIS1K9fP1WqVMnW3rRpU3Xq1Enr16+3tVWoUEHff/+9zpw5k+2yzNpOAPBXjLGMsXkpqjF28eLFatmypWrXri1JKl++vLp06WJ3inlmZqbWrl2rrl27qmXLllmWcfPSglWrVikkJEQ9evTIsc+tup19Zffu3Tp69KheeOEFVahQIcd6IiMjdebMGdv/qfTndnF3d9ejjz5aoLphDkI37jjVqlWTq6trlvZffvlFPXr0kJeXlzw9PVWlShXbDWLyc91rjRo17KZv/nHw+++/3/K8N+e/Oe+5c+d0/fp128Dxv7Jry86JEydsAe7mNWQdOnSQlHX93NzcspxS97/1SNLx48fl5+encuXK2fWrV69evurJzW+//SbDMFSnTh1VqVLF7rV//36dO3dOklSzZk3FxMRozpw58vb2VkREhGbMmHHb1ynn9f8h/bn+t/P/kZfjx49Lyn57NmjQQBcuXFBqaqokadKkSdq7d68CAgLUunVrjR492u6PF7O2EwD8FWMsY2xeimKMvXz5stavX68OHTro0KFDtlfbtm31448/6tdff5UknT9/XikpKWrcuHGuyzt8+HCefW7V7ewrhw8flqQ8a+rUqZP8/PxsXzRkZmZq6dKl6tatG3dxL2a4pht3nP/9BvGmy5cvq0OHDvL09NTYsWMVHBwsNzc37dq1S6+++qoyMzPzXG5Od+g08vHUvduZNz8yMjLUqVMnXbp0Sa+++qrq168vDw8PnT59Wv369cuyfo6+22hmZqYsFos+//zzbGv53z9CpkyZon79+unjjz/Wl19+qeeee05xcXH67rvvVL169QJ9vtn/H4WtV69eateundasWaMvv/xSkydP1ltvvaXVq1fbjtKYsZ0A4K8YYxlj81IUY+yKFSuUlpamKVOmaMqUKVneX7x4scaMGVNonyflfMQ7uxugSebtK//L2dlZjz/+uGbPnq33339f33zzjc6cOZPnUwdQ9AjdKBW2bNmiixcvavXq1Wrfvr2t/ejRow6s6r+qVq0qNzc3HTp0KMt72bX91Z49e/Trr79q4cKFioyMtLVv3LixwDUFBgYqPj5eV69etRugDx48WOBl3hQcHCzDMFSzZk3VrVs3z/5NmjRRkyZN9MYbb+jbb79V27ZtNWvWLL355puSCn7qV24CAwML/P+R3+VL2W/PAwcOyNvbWx4eHrY2Pz8/DR48WIMHD9a5c+fUvHlzjR8/3u7UyLy2EwCYgTH21jHG3t4Yu3jxYjVu3FixsbFZ3vvggw+0ZMkSjRkzRlWqVJGnp6f27t2b6/KCg4Pz7HPz7Iu/3mH+5plr+ZHffeXm5RJ79+5VeHh4rsuMjIzUlClT9Mknn+jzzz9XlSpVFBERke+aUDQ4vRylws1vXf/3W9b09HS9//77jirJjrOzs8LDw7V27Vq763YPHTqkzz//PF/zS/brZxiG7bESBdG5c2f98ccfmjlzpq0tIyND06dPL/Ayb3rkkUfk7OysMWPGZPnm2zAMXbx4UZKUkpKiP/74w+79Jk2ayMnJSWlpabY2Dw+PQn3MiiRFRERo+/btSkhIsLVdunQpx8eR3Co/Pz+FhoZq4cKFdrXv3btXX375pTp37izpz23+11P9qlatKn9/f9s2yO92AgAzMMbeOsbYgo+xJ0+e1NatW9WrVy/17Nkzyys6OlqHDh3S999/LycnJ3Xv3l2ffPKJfvzxxyzLurl9Hn30Uf30009as2ZNjn1uBuGtW7fa3svIyNCHH36Y7/XO777SvHlz1axZU9OmTcuy7f/6f9q0aVM1bdpUc+bM0apVq9SnTx+eZ18M8T+CUuHuu+9WxYoVFRUVpeeee04Wi0WLFi0qVqcTjx49Wl9++aXatm2rQYMGKSMjQ++9954aN25sNyhlp379+goODtbLL7+s06dPy9PTU6tWrcrXtXA56dq1q9q2bavXXntNx44dU8OGDbV69epCuU44ODhYb775poYPH65jx46pe/fuKl++vI4ePao1a9Zo4MCBevnll7V582YNHTpUjz32mOrWras//vhDixYtkrOzs90NQlq0aKFNmzZp6tSp8vf3V82aNRUWFnZbNb7yyiv66KOP1KlTJw0bNsz2OJMaNWro0qVL+f7mf+rUqSpbtqxdm5OTk15//XVNnjxZDz30kNq0aaP+/fvbHhnm5eWl0aNHS/rzJivVq1dXz549FRISonLlymnTpk364YcfbKfU5Xc7AYAZGGNvHWNswcfYJUuWyDAMPfzww9m+37lzZ7m4uGjx4sUKCwvThAkT9OWXX6pDhw4aOHCgGjRooLNnz2rFihXatm2bKlSooH/84x9auXKlHnvsMT311FNq0aKFLl26pHXr1mnWrFkKCQlRo0aNdNddd2n48OG6dOmSKlWqpGXLlmX54iI3+d1XnJycNHPmTHXt2lWhoaGKjo6Wn5+fDhw4oF9++UVffPGFXf/IyEi9/PLLksSp5cVVUdwiHTBDTo8zadSoUbb9v/nmG+Ouu+4y3N3dDX9/f+OVV14xvvjiiyyPecjpcSaTJ0/OskxJRmxsrG06p8eZZPe4p8DAwCyP4IiPjzeaNWtmuLq6GsHBwcacOXOMl156yXBzc8thK/zXvn37jPDwcKNcuXKGt7e3MWDAANtjU/73sRFRUVGGh4dHlvmzq/3ixYvGk08+aXh6ehpeXl7Gk08+aXvEyO08zuSmVatWGffcc4/h4eFheHh4GPXr1zeGDBliHDx40DAMwzhy5Ijx1FNPGcHBwYabm5tRqVIl49577zU2bdpkt5wDBw4Y7du3N9zd3Q1Jtu2a0+NMunTpkqXGDh06GB06dLBr2717t9GuXTvDarUa1atXN+Li4ox3333XkGQkJibmus43t2d2L2dnZ1u/TZs2GW3btjXc3d0NT09Po2vXrsa+ffts76elpRn/+Mc/jJCQEKN8+fKGh4eHERISYrz//vu2PvndTgCQX4yx9hhji88Y26RJE6NGjRq5bpOOHTsaVatWNW7cuGEYhmEcP37ciIyMNKpUqWJYrVajVq1axpAhQ4y0tDTbPBcvXjSGDh1qVKtWzXB1dTWqV69uREVFGRcuXLD1OXz4sBEeHm5YrVbDx8fHeP31142NGzdm+8iw291XDMMwtm3bZnTq1Mk2/jdt2tTuUXg3nT171nB2djbq1q2b63aB41gMoxh9DQkgi+7du+uXX37Rb7/95uhSIOmFF17QBx98oKtXrzr8ZjkAgNvDGFu8MMYWzIULF+Tn56dRo0Zp5MiRji4H2eCabqAYuX79ut30b7/9pvXr16tjx46OKaiU++v/x8WLF7Vo0SLdc889/DEAACUMY2zxwhhbeBYsWKCMjAw9+eSTji4FOeBIN1CM+Pn5qV+/fqpVq5aOHz+umTNnKi0tTbt371adOnUcXV6pExoaqo4dO6pBgwZKSkrS3LlzdebMGcXHx9vddRQAUPwxxhYvjLG3b/Pmzdq3b59Gjhype++9V6tXr3Z0ScgBoRsoRqKjo/XVV18pMTFRVqtVbdq00YQJE9S8eXNHl1Yqvf7661q5cqVOnToli8Wi5s2bKzY2Ns/HdwAAih/G2OKFMfb2dezY0faYt48++kjVqlVzdEnIAaEbAADkaevWrZo8ebJ27typs2fPas2aNerevXuu82zZskUxMTH65ZdfFBAQoDfeeEP9+vUrknoBACguuKYbAADkKTU1VSEhIZoxY0a++h89elRdunTRvffeq4SEBL3wwgt6+umnszzqBgCAOx1HugEAwC2xWCx5Hul+9dVX9dlnn2nv3r22tj59+ujy5cvasGFDEVQJAEDx4OLoAopaZmamzpw5o/Lly8tisTi6HAAAsjAMQ1euXJG/v7+cnErmSWnbt2/Pcm1mRESEXnjhhRznSUtLU1pamm06MzNTly5dUuXKlRmzAQDFTn7H61IXus+cOaOAgABHlwEAQJ5Onjyp6tWrO7qMAklMTJSPj49dm4+Pj1JSUnT9+nW5u7tnmScuLk5jxowpqhIBACgUeY3XpS50ly9fXtKfG8bT09PB1QAAkFVKSooCAgJsY1ZpMXz4cMXExNimk5OTVaNGDcZsAECxlN/xutSF7punp3l6ejKAAwCKtZJ8SrWvr6+SkpLs2pKSkuTp6ZntUW5JslqtslqtWdoZswEAxVle43XJvFAMAAAUa23atFF8fLxd28aNG9WmTRsHVQQAgGMQugEAQJ6uXr2qhIQEJSQkSPrzkWAJCQk6ceKEpD9PDY+MjLT1f/bZZ3XkyBG98sorOnDggN5//339+9//1osvvuiI8gEAcBhCNwAAyNOPP/6oZs2aqVmzZpKkmJgYNWvWTKNGjZIknT171hbAJalmzZr67LPPtHHjRoWEhGjKlCmaM2eOIiIiHFI/AACOUuqe052SkiIvLy8lJydzfRgAoFhirPoT2wEAUJzld5ziSDcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBIXRxdwJ7BYHF0BkH+G4egKAAAAgNKjWBzpnjFjhoKCguTm5qawsDDt2LEjx74LFiyQxWKxe7m5uRVhtQAAAAAA5I/DQ/fy5csVExOj2NhY7dq1SyEhIYqIiNC5c+dynMfT01Nnz561vY4fP16EFQMAAAAAkD8OD91Tp07VgAEDFB0drYYNG2rWrFkqW7as5s2bl+M8FotFvr6+tpePj08RVgwAAAAAQP44NHSnp6dr586dCg8Pt7U5OTkpPDxc27dvz3G+q1evKjAwUAEBAerWrZt++eWXoigXAAAAAIBb4tDQfeHCBWVkZGQ5Uu3j46PExMRs56lXr57mzZunjz/+WB999JEyMzN1991369SpU9n2T0tLU0pKit0LAAAAAICi4PDTy29VmzZtFBkZqdDQUHXo0EGrV69WlSpV9MEHH2TbPy4uTl5eXrZXQEBAEVcMAAAAACitHBq6vb295ezsrKSkJLv2pKQk+fr65msZZcqUUbNmzXTo0KFs3x8+fLiSk5Ntr5MnT9523QAAAAAA5IdDQ7erq6tatGih+Ph4W1tmZqbi4+PVpk2bfC0jIyNDe/bskZ+fX7bvW61WeXp62r0AAAAAACgKLo4uICYmRlFRUWrZsqVat26tadOmKTU1VdHR0ZKkyMhIVatWTXFxcZKksWPH6q677lLt2rV1+fJlTZ48WcePH9fTTz/tyNUAAAAAACALh4fu3r176/z58xo1apQSExMVGhqqDRs22G6uduLECTk5/feA/O+//64BAwYoMTFRFStWVIsWLfTtt9+qYcOGjloFAAAAAACyZTEMw3B0EUUpJSVFXl5eSk5OLrRTzS2WQlkMUCRK1x4PlExmjFUlEdsBAFCc5XecKnF3LwcAAAAAoKQgdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACZxcXQBAAAAAEqwJRZHVwDcmseNIv04jnQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEhdHFwAAAFAULBZHVwDkn2E4ugIAhYUj3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AACBfZsyYoaCgILm5uSksLEw7duzItf+0adNUr149ubu7KyAgQC+++KL+85//FFG1AAAUD4RuAACQp+XLlysmJkaxsbHatWuXQkJCFBERoXPnzmXbf8mSJXrttdcUGxur/fv3a+7cuVq+fLlef/31Iq4cAADHInQDAIA8TZ06VQMGDFB0dLQaNmyoWbNmqWzZspo3b162/b/99lu1bdtWjz/+uIKCgvTAAw+ob9++eR4dBwDgTkPoBgAAuUpPT9fOnTsVHh5ua3NyclJ4eLi2b9+e7Tx33323du7caQvZR44c0fr169W5c+ccPyctLU0pKSl2LwAASjoXRxcAAACKtwsXLigjI0M+Pj527T4+Pjpw4EC28zz++OO6cOGC7rnnHhmGoT/++EPPPvtsrqeXx8XFacyYMYVaOwAAjsaRbgAAUOi2bNmiCRMm6P3339euXbu0evVqffbZZxo3blyO8wwfPlzJycm218mTJ4uwYgAAzMGRbgAAkCtvb285OzsrKSnJrj0pKUm+vr7ZzjNy5Eg9+eSTevrppyVJTZo0UWpqqgYOHKgRI0bIySnr9/5Wq1VWq7XwVwAAAAfiSDcAAMiVq6urWrRoofj4eFtbZmam4uPj1aZNm2znuXbtWpZg7ezsLEkyDMO8YgEAKGY40g0AAPIUExOjqKgotWzZUq1bt9a0adOUmpqq6OhoSVJkZKSqVaumuLg4SVLXrl01depUNWvWTGFhYTp06JBGjhyprl272sI3AAClAaEbAADkqXfv3jp//rxGjRqlxMREhYaGasOGDbabq504ccLuyPYbb7whi8WiN954Q6dPn1aVKlXUtWtXjR8/3lGrAACAQ1iMUnaOV0pKiry8vJScnCxPT89CWabFUiiLAYpE6drjgZLJjLGqJCrs7cB4jZKkRI3XS9i5UMI8Xjg7WH7HKY50Ayi+GMRRkhTSAA4AAO4s3EgNAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMUi9A9Y8YMBQUFyc3NTWFhYdqxY0e+5lu2bJksFou6d+9uboEAAAAAABSAw0P38uXLFRMTo9jYWO3atUshISGKiIjQuXPncp3v2LFjevnll9WuXbsiqhQAAAAAgFvj8NA9depUDRgwQNHR0WrYsKFmzZqlsmXLat68eTnOk5GRoSeeeEJjxoxRrVq1irBaAAAAAADyz6GhOz09XTt37lR4eLitzcnJSeHh4dq+fXuO840dO1ZVq1ZV//798/yMtLQ0paSk2L0AAAAAACgKDg3dFy5cUEZGhnx8fOzafXx8lJiYmO0827Zt09y5czV79ux8fUZcXJy8vLxsr4CAgNuuGwAAAACA/HD46eW34sqVK3ryySc1e/ZseXt752ue4cOHKzk52fY6efKkyVUCAAAAAPAnF0d+uLe3t5ydnZWUlGTXnpSUJF9f3yz9Dx8+rGPHjqlr1662tszMTEmSi4uLDh48qODgYLt5rFarrFarCdUDAAAAAJA7hx7pdnV1VYsWLRQfH29ry8zMVHx8vNq0aZOlf/369bVnzx4lJCTYXg8//LDuvfdeJSQkcOo4AAAAAKBYceiRbkmKiYlRVFSUWrZsqdatW2vatGlKTU1VdHS0JCkyMlLVqlVTXFyc3Nzc1LhxY7v5K1SoIElZ2gEAAAAAcDSHh+7evXvr/PnzGjVqlBITExUaGqoNGzbYbq524sQJOTmVqEvPAQAAAACQJFkMwzAcXURRSklJkZeXl5KTk+Xp6Vkoy7RYCmUxQJEoUXv8EnYulCCPF97OZcZYVRIV9nZgvEZJwngNmKiQxuz8jlMcQgYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAD5MmPGDAUFBcnNzU1hYWHasWNHrv0vX76sIUOGyM/PT1arVXXr1tX69euLqFoAAIoHF0cXAAAAir/ly5crJiZGs2bNUlhYmKZNm6aIiAgdPHhQVatWzdI/PT1dnTp1UtWqVbVy5UpVq1ZNx48fV4UKFYq+eAAAHIjQDQAA8jR16lQNGDBA0dHRkqRZs2bps88+07x58/Taa69l6T9v3jxdunRJ3377rcqUKSNJCgoKKsqSAQAoFji9HAAA5Co9PV07d+5UeHi4rc3JyUnh4eHavn17tvOsW7dObdq00ZAhQ+Tj46PGjRtrwoQJysjIKKqyAQAoFjjSDQAAcnXhwgVlZGTIx8fHrt3Hx0cHDhzIdp4jR45o8+bNeuKJJ7R+/XodOnRIgwcP1o0bNxQbG5vtPGlpaUpLS7NNp6SkFN5KAADgIBzpBgAAhS4zM1NVq1bVhx9+qBYtWqh3794aMWKEZs2aleM8cXFx8vLysr0CAgKKsGIAAMxB6AYAALny9vaWs7OzkpKS7NqTkpLk6+ub7Tx+fn6qW7eunJ2dbW0NGjRQYmKi0tPTs51n+PDhSk5Otr1OnjxZeCsBAICDELoBAECuXF1d1aJFC8XHx9vaMjMzFR8frzZt2mQ7T9u2bXXo0CFlZmba2n799Vf5+fnJ1dU123msVqs8PT3tXgAAlHSEbgAAkKeYmBjNnj1bCxcu1P79+zVo0CClpqba7mYeGRmp4cOH2/oPGjRIly5d0vPPP69ff/1Vn332mSZMmKAhQ4Y4ahUAAHAIbqQGAADy1Lt3b50/f16jRo1SYmKiQkNDtWHDBtvN1U6cOCEnp/9+lx8QEKAvvvhCL774opo2bapq1arp+eef16uvvuqoVQAAwCEshmEYji6iKKWkpMjLy0vJycmFdtqaxVIoiwGKRIna45ewc6EEebzwdi4zxqqSqLC3A+M1ShLGa8BEhTRm53ec4vRyAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATFIsQveMGTMUFBQkNzc3hYWFaceOHTn2Xb16tVq2bKkKFSrIw8NDoaGhWrRoURFWCwAAAABA/jg8dC9fvlwxMTGKjY3Vrl27FBISooiICJ07dy7b/pUqVdKIESO0fft2/fzzz4qOjlZ0dLS++OKLIq4cAAAAAIDcOTx0T506VQMGDFB0dLQaNmyoWbNmqWzZspo3b162/Tt27KgePXqoQYMGCg4O1vPPP6+mTZtq27ZtRVw5AADFW1BQkMaOHasTJ044uhQAAEoth4bu9PR07dy5U+Hh4bY2JycnhYeHa/v27XnObxiG4uPjdfDgQbVv397MUgEAKHFeeOEFrV69WrVq1VKnTp20bNkypaWlObosAABKFYeG7gsXLigjI0M+Pj527T4+PkpMTMxxvuTkZJUrV06urq7q0qWLpk+frk6dOmXbNy0tTSkpKXYvAABKgxdeeEEJCQnasWOHGjRooGHDhsnPz09Dhw7Vrl27HF0eAAClgsNPLy+I8uXLKyEhQT/88IPGjx+vmJgYbdmyJdu+cXFx8vLysr0CAgKKtlgAABysefPmevfdd3XmzBnFxsZqzpw5atWqlUJDQzVv3jwZhuHoEgEAuGO5OPLDvb295ezsrKSkJLv2pKQk+fr65jifk5OTateuLUkKDQ3V/v37FRcXp44dO2bpO3z4cMXExNimU1JSCN4AgFLlxo0bWrNmjebPn6+NGzfqrrvuUv/+/XXq1Cm9/vrr2rRpk5YsWeLoMgEAuCM5NHS7urqqRYsWio+PV/fu3SVJmZmZio+P19ChQ/O9nMzMzByvUbNarbJarYVRLgAAJcquXbs0f/58LV26VE5OToqMjNQ777yj+vXr2/r06NFDrVq1cmCVAADc2RwauiUpJiZGUVFRatmypVq3bq1p06YpNTVV0dHRkqTIyEhVq1ZNcXFxkv48Xbxly5YKDg5WWlqa1q9fr0WLFmnmzJmOXA0AAIqdVq1aqVOnTpo5c6a6d++uMmXKZOlTs2ZN9enTxwHVAQBQOjg8dPfu3Vvnz5/XqFGjlJiYqNDQUG3YsMF2c7UTJ07Iyem/l56npqZq8ODBOnXqlNzd3VW/fn199NFH6t27t6NWAQCAYunIkSMKDAzMtY+Hh4fmz59fRBUBAFD6WIxSdveUlJQUeXl5KTk5WZ6enoWyTIulUBYDFIkStccvYedCCfJ44e1chTVW/fDDD8rMzFRYWJhd+/fffy9nZ2e1bNnydks1VWGP2YzXKEkYrwETFdKYnd9xqkTevRwAAORtyJAhOnnyZJb206dPa8iQIQ6oCACA0ofQDQDAHWrfvn1q3rx5lvZmzZpp3759DqgIAIDSh9ANAMAdymq1ZnkspySdPXtWLi4Ov60LAAClAqEbAIA71AMPPKDhw4crOTnZ1nb58mW9/vrr6tSpkwMrAwCg9OBrbgAA7lBvv/222rdvr8DAQDVr1kySlJCQIB8fHy1atMjB1QEAUDoQugEAuENVq1ZNP//8sxYvXqyffvpJ7u7uio6OVt++fbN9ZjcAACh8hG4AAO5gHh4eGjhwoKPLAACg1CJ0AwBwh9u3b59OnDih9PR0u/aHH37YQRUBAFB6FCh0nzx5UhaLRdWrV5ck7dixQ0uWLFHDhg35Nh0AgGLiyJEj6tGjh/bs2SOLxSLDMCRJFotFkpSRkeHI8gAAKBUKdPfyxx9/XF999ZUkKTExUZ06ddKOHTs0YsQIjR07tlALBAAABfP888+rZs2aOnfunMqWLatffvlFW7duVcuWLbVlyxZHlwcAQKlQoNC9d+9etW7dWpL073//W40bN9a3336rxYsXa8GCBYVZHwAAKKDt27dr7Nix8vb2lpOTk5ycnHTPPfcoLi5Ozz33nKPLAwCgVChQ6L5x44asVqskadOmTbZrwurXr6+zZ88WXnUAAKDAMjIyVL58eUmSt7e3zpw5I0kKDAzUwYMHHVkaAAClRoFCd6NGjTRr1ix9/fXX2rhxox588EFJ0pkzZ1S5cuVCLRAAABRM48aN9dNPP0mSwsLCNGnSJH3zzTcaO3asatWq5eDqAAAoHQoUut966y198MEH6tixo/r27auQkBBJ0rp162ynnQMAAMd64403lJmZKUkaO3asjh49qnbt2mn9+vV69913HVwdAAClQ4HuXt6xY0dduHBBKSkpqlixoq194MCBKlu2bKEVBwAACi4iIsL279q1a+vAgQO6dOmSKlasaLuDOQAAMFeBjnRfv35daWlptsB9/PhxTZs2TQcPHlTVqlULtUAAAHDrbty4IRcXF+3du9euvVKlSgRuAACKUIFCd7du3fSvf/1LknT58mWFhYVpypQp6t69u2bOnFmoBQIAgFtXpkwZ1ahRg2dxAwDgYAUK3bt27VK7du0kSStXrpSPj4+OHz+uf/3rX1wjBgBAMTFixAi9/vrrunTpkqNLAQCg1CrQNd3Xrl2zPYLkyy+/1COPPCInJyfdddddOn78eKEWCAAACua9997ToUOH5O/vr8DAQHl4eNi9v2vXLgdVBgBA6VGg0F27dm2tXbtWPXr00BdffKEXX3xRknTu3Dl5enoWaoEAAKBgunfv7ugSAAAo9QoUukeNGqXHH39cL774ou677z61adNG0p9HvZs1a1aoBQIAgIKJjY11dAkAAJR6BQrdPXv21D333KOzZ8/antEtSffff7969OhRaMUBAAAAAFCSFSh0S5Kvr698fX116tQpSVL16tXVunXrQisMAADcHicnp1wfD8adzQEAMF+BQndmZqbefPNNTZkyRVevXpUklS9fXi+99JJGjBghJ6cC3RQdAAAUojVr1thN37hxQ7t379bChQs1ZswYB1UFAEDpUqDQPWLECM2dO1cTJ05U27ZtJUnbtm3T6NGj9Z///Efjx48v1CIBAMCt69atW5a2nj17qlGjRlq+fLn69+/vgKoAAChdChS6Fy5cqDlz5ujhhx+2tTVt2lTVqlXT4MGDCd0AABRjd911lwYOHOjoMgAAKBUKdB74pUuXVL9+/Szt9evX16VLl267KAAAYI7r16/r3XffVbVq1RxdCgAApUKBjnSHhITovffe07vvvmvX/t5776lp06aFUhgAALg9FStWtLuRmmEYunLlisqWLauPPvrIgZUBAFB6FCh0T5o0SV26dNGmTZtsz+jevn27Tp48qfXr1xdqgQAAoGDeeecdu9Dt5OSkKlWqKCwsTBUrVnRgZQAAlB4FCt0dOnTQr7/+qhkzZujAgQOSpEceeUQDBw7Um2++qXbt2hVqkQAA4Nb169fP0SUAAFDqFfg53f7+/llumPbTTz9p7ty5+vDDD2+7MAAAcHvmz5+vcuXK6bHHHrNrX7Fiha5du6aoqCgHVQYAQOnBA7UBALhDxcXFydvbO0t71apVNWHCBAdUBABA6UPoBgDgDnXixAnVrFkzS3tgYKBOnDjhgIoAACh9CN0AANyhqlatqp9//jlL+08//aTKlSs7oCIAAEqfW7qm+5FHHsn1/cuXL99OLQAAoBD17dtXzz33nMqXL6/27dtLkv7v//5Pzz//vPr06ePg6gAAKB1uKXR7eXnl+X5kZORtFQQAAArHuHHjdOzYMd1///1ycflzyM/MzFRkZCTXdAMAUERuKXTPnz/frDoAAEAhc3V11fLly/Xmm28qISFB7u7uatKkiQIDAx1dGgAApUaBHxkGAABKhjp16qhOnTqOLgMAgFKJG6kBAHCHevTRR/XWW29laZ80aVKWZ3cDAABzELoBALhDbd26VZ07d87S/tBDD2nr1q0OqAgAgNKH0A0AwB3q6tWrcnV1zdJepkwZpaSkOKAiAABKH0I3AAB3qCZNmmj58uVZ2pctW6aGDRs6oCIAAEofbqQGAMAdauTIkXrkkUd0+PBh3XfffZKk+Ph4LVmyRCtXrnRwdQAAlA6EbgAA7lBdu3bV2rVrNWHCBK1cuVLu7u4KCQnR5s2bValSJUeXBwBAqUDoBgDgDtalSxd16dJFkpSSkqKlS5fq5Zdf1s6dO5WRkeHg6gAAuPNxTTcAAHe4rVu3KioqSv7+/poyZYruu+8+fffdd44uCwCAUoEj3QAA3IESExO1YMECzZ07VykpKerVq5fS0tK0du1abqIGAEAR4kg3AAB3mK5du6pevXr6+eefNW3aNJ05c0bTp093dFkAAJRKHOkGAOAO8/nnn+u5557ToEGDVKdOHUeXAwBAqcaRbgAA7jDbtm3TlStX1KJFC4WFhem9997ThQsXHF0WAAClEqEbAIA7zF133aXZs2fr7NmzeuaZZ7Rs2TL5+/srMzNTGzdu1JUrVxxdIgAApQahGwCAO5SHh4eeeuopbdu2TXv27NFLL72kiRMnqmrVqnr44YcdXR4AAKUCoRsAgFKgXr16mjRpkk6dOqWlS5c6uhwAAEoNQjcAAKWIs7OzunfvrnXr1jm6FAAASgVCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAMiXGTNmKCgoSG5ubgoLC9OOHTvyNd+yZctksVjUvXt3cwsEAKAYInQDAIA8LV++XDExMYqNjdWuXbsUEhKiiIgInTt3Ltf5jh07ppdfflnt2rUrokoBACheCN0AACBPU6dO1YABAxQdHa2GDRtq1qxZKlu2rObNm5fjPBkZGXriiSc0ZswY1apVqwirBQCg+CB0AwCAXKWnp2vnzp0KDw+3tTk5OSk8PFzbt2/Pcb6xY8eqatWq6t+/f1GUCQBAseTi6AIAAEDxduHCBWVkZMjHx8eu3cfHRwcOHMh2nm3btmnu3LlKSEjI9+ekpaUpLS3NNp2SklKgegEAKE440g0AAArVlStX9OSTT2r27Nny9vbO93xxcXHy8vKyvQICAkysEgCAosGRbgAAkCtvb285OzsrKSnJrj0pKUm+vr5Z+h8+fFjHjh1T165dbW2ZmZmSJBcXFx08eFDBwcFZ5hs+fLhiYmJs0ykpKQRvAECJR+gGAAC5cnV1VYsWLRQfH2977FdmZqbi4+M1dOjQLP3r16+vPXv22LW98cYbunLliv75z3/mGKStVqusVmuh1w8AgCMRugEAQJ5iYmIUFRWlli1bqnXr1po2bZpSU1MVHR0tSYqMjFS1atUUFxcnNzc3NW7c2G7+ChUqSFKWdgAA7nSEbgAAkKfevXvr/PnzGjVqlBITExUaGqoNGzbYbq524sQJOTlxqxgAAP7KYhiG4egiilJKSoq8vLyUnJwsT0/PQlmmxVIoiwGKRIna45ewc6EEebzwdi4zxqqSqLC3A+M1ShLGa8BEhTRm53ecKhZfSc+YMUNBQUFyc3NTWFiYduzYkWPf2bNnq127dqpYsaIqVqyo8PDwXPsDAAAAAOAoDg/dy5cvV0xMjGJjY7Vr1y6FhIQoIiJC586dy7b/li1b1LdvX3311Vfavn27AgIC9MADD+j06dNFXDkAAAAAALlzeOieOnWqBgwYoOjoaDVs2FCzZs1S2bJlNW/evGz7L168WIMHD1ZoaKjq16+vOXPm2O6gCgAAAABAceLQ0J2enq6dO3cqPDzc1ubk5KTw8HBt3749X8u4du2abty4oUqVKmX7flpamlJSUuxeAAAAAAAUBYeG7gsXLigjI8N259ObfHx8lJiYmK9lvPrqq/L397cL7v8rLi5OXl5etldOzwYFAAAAAKCwOfz08tsxceJELVu2TGvWrJGbm1u2fYYPH67k5GTb6+TJk0VcJQAAAACgtHLoc7q9vb3l7OyspKQku/akpCT5+vrmOu/bb7+tiRMnatOmTWratGmO/axWq6xWa6HUCwAAAADArXDokW5XV1e1aNHC7iZoN2+K1qZNmxznmzRpksaNG6cNGzaoZcuWRVEqAAAAAAC3zKFHuiUpJiZGUVFRatmypVq3bq1p06YpNTVV0dHRkqTIyEhVq1ZNcXFxkqS33npLo0aN0pIlSxQUFGS79rtcuXIqV66cw9YDAAAAAIC/cnjo7t27t86fP69Ro0YpMTFRoaGh2rBhg+3maidOnJCT038PyM+cOVPp6enq2bOn3XJiY2M1evTooiwdAAAAAIBcOTx0S9LQoUM1dOjQbN/bsmWL3fSxY8fMLwgAAAAAgEJQou9eDgAAAABAcUboBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAD5MmPGDAUFBcnNzU1hYWHasWNHjn1nz56tdu3aqWLFiqpYsaLCw8Nz7Q8AwJ2K0A0AAPK0fPlyxcTEKDY2Vrt27VJISIgiIiJ07ty5bPtv2bJFffv21VdffaXt27crICBADzzwgE6fPl3ElQMA4FiEbgAAkKepU6dqwIABio6OVsOGDTVr1iyVLVtW8+bNy7b/4sWLNXjwYIWGhqp+/fqaM2eOMjMzFR8fX8SVAwDgWIRuAACQq/T0dO3cuVPh4eG2NicnJ4WHh2v79u35Wsa1a9d048YNVapUKcc+aWlpSklJsXsBAFDSEboBAECuLly4oIyMDPn4+Ni1+/j4KDExMV/LePXVV+Xv728X3P8qLi5OXl5etldAQMBt1Q0AQHHg8NB9Kzdl+eWXX/Too48qKChIFotF06ZNK7pCAQBAgUycOFHLli3TmjVr5ObmlmO/4cOHKzk52fY6efJkEVYJAIA5HBq6b/WmLNeuXVOtWrU0ceJE+fr6FnG1AACUTt7e3nJ2dlZSUpJde1JSUp7j8dtvv62JEyfqyy+/VNOmTXPta7Va5enpafcCAKCkc2jovtWbsrRq1UqTJ09Wnz59ZLVai7haAABKJ1dXV7Vo0cLuJmg3b4rWpk2bHOebNGmSxo0bpw0bNqhly5ZFUSoAAMWOw0J3YdyUBQAAFI2YmBjNnj1bCxcu1P79+zVo0CClpqYqOjpakhQZGanhw4fb+r/11lsaOXKk5s2bp6CgICUmJioxMVFXr1511CoAAOAQLo764NxuynLgwIFC+5y0tDSlpaXZprkTKgAAt6537946f/68Ro0apcTERIWGhmrDhg22cfzEiRNycvrvd/kzZ85Uenq6evbsabec2NhYjR49uihLBwDAoRwWuotKXFycxowZ4+gyAAAo8YYOHaqhQ4dm+96WLVvspo8dO2Z+QQAAlAAOO738dm7Kciu4EyoAAAAAwFEcFroLelOWW8WdUAEAAAAAjuLQ08tjYmIUFRWlli1bqnXr1po2bVqWm7JUq1ZNcXFxkv68+dq+ffts/z59+rQSEhJUrlw51a5d22HrAQAAAABAdhwaum/1pixnzpxRs2bNbNNvv/223n77bXXo0CHLtWQAAAAAADiaw2+kdis3ZQkKCpJhGEVQFQAAAAAAt89h13QDAAAAAHCnI3QDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYpFqF7xowZCgoKkpubm8LCwrRjx45c+69YsUL169eXm5ubmjRpovXr1xdRpQAAlF6M1wAA3DqHh+7ly5crJiZGsbGx2rVrl0JCQhQREaFz585l2//bb79V37591b9/f+3evVvdu3dX9+7dtXfv3iKuHACA0oPxGgCAgrEYhmE4soCwsDC1atVK7733niQpMzNTAQEBGjZsmF577bUs/Xv37q3U1FR9+umntra77rpLoaGhmjVrVp6fl5KSIi8vLyUnJ8vT07NQ1sFiKZTFAEXCsXv8LVrCzoUS5PHC27nMGKtuV1GP11LhbwfGa5QkjNeAiQppzM7vOOVSKJ9WQOnp6dq5c6eGDx9ua3NyclJ4eLi2b9+e7Tzbt29XTEyMXVtERITWrl2bbf+0tDSlpaXZppOTkyX9uYGA0qhE/ehfc3QBwC0oxJ3r5hjl4O/FbYpivJYYs4H/VaJ+7BmvUdIU0g6W3/HaoaH7woULysjIkI+Pj127j4+PDhw4kO08iYmJ2fZPTEzMtn9cXJzGjBmTpT0gIKCAVQMlm5eXoysA7lADCn/nunLliryKwU5bFOO1xJgN/K9isOsDd65CHrPzGq8dGrqLwvDhw+2+ac/MzNSlS5dUuXJlWTjPrNhKSUlRQECATp48WWxOrQTuBOxbJYNhGLpy5Yr8/f0dXUqRYswuefidApiH/av4y+947dDQ7e3tLWdnZyUlJdm1JyUlydfXN9t5fH19b6m/1WqV1Wq1a6tQoULBi0aR8vT05JcMYAL2reKvOBzhvqkoxmuJMbsk43cKYB72r+ItP+O1Q+9e7urqqhYtWig+Pt7WlpmZqfj4eLVp0ybbedq0aWPXX5I2btyYY38AAHB7GK8BACg4h59eHhMTo6ioKLVs2VKtW7fWtGnTlJqaqujoaElSZGSkqlWrpri4OEnS888/rw4dOmjKlCnq0qWLli1bph9//FEffvihI1cDAIA7GuM1AAAF4/DQ3bt3b50/f16jRo1SYmKiQkNDtWHDBtvNV06cOCEnp/8ekL/77ru1ZMkSvfHGG3r99ddVp04drV27Vo0bN3bUKsAEVqtVsbGxWU4zBHB72LdQUIzXyA6/UwDzsH/dORz+nG4AAAAAAO5UDr2mGwAAAACAOxmhGwAAAAAAkxC6AQAAAAAwCaEbRSYoKEjTpk1zdBkAANzRGG8BoHghdCMLi8WS62v06NEFWu4PP/yggQMH3lZtHTt21AsvvHBbywCKA7P2s5vLXrt2baH1A2CO4jze3rR06VI5OztryJAhhbI8oKgVh/H2pmeeeUbOzs5asWJFgT8TJZPDHxmG4ufs2bO2fy9fvlyjRo3SwYMHbW3lypWz/dswDGVkZMjFJe8fpSpVqhRuoUAJdiv7GYA7U0kYb+fOnatXXnlFH3zwgaZMmSI3N7dCW/atSk9Pl6urq8M+HyVTcRlvr127pmXLlumVV17RvHnz9NhjjxXJ5+aE/alocaQbWfj6+tpeXl5eslgstukDBw6ofPny+vzzz9WiRQtZrVZt27ZNhw8fVrdu3eTj46Ny5cqpVatW2rRpk91y/3q6m8Vi0Zw5c9SjRw+VLVtWderU0bp1626r9lWrVqlRo0ayWq0KCgrSlClT7N5///33VadOHbm5ucnHx0c9e/a0vbdy5Uo1adJE7u7uqly5ssLDw5Wamnpb9QA5yW0/8/X11bJly9SgQQO5ubmpfv36ev/9923zpqena+jQofLz85Obm5sCAwMVFxcn6c/9TJJ69Oghi8Vim75VmZmZGjt2rKpXry6r1Wp7JnN+ajAMQ6NHj1aNGjVktVrl7++v5557rmAbCriDFffx9ujRo/r222/12muvqW7dulq9enWWPvPmzbONu35+fho6dKjtvcuXL+uZZ56Rj4+P3Nzc1LhxY3366aeSpNGjRys0NNRuWdOmTbP7ndWvXz91795d48ePl7+/v+rVqydJWrRokVq2bKny5cvL19dXjz/+uM6dO2e3rF9++UV/+9vf5OnpqfLly6tdu3Y6fPiwtm7dqjJlyigxMdGu/wsvvKB27drluU1Q8hSX8XbFihVq2LChXnvtNW3dulUnT560ez8tLU2vvvqqAgICZLVaVbt2bc2dO9f2fk4/01L2Z4J2795d/fr1s00HBQVp3LhxioyMlKenp+1smFdffVV169ZV2bJlVatWLY0cOVI3btywW9Ynn3yiVq1ayc3NTd7e3urRo4ckaezYsWrcuHGWdQ0NDdXIkSNz3R6lDaEbBfLaa69p4sSJ2r9/v5o2baqrV6+qc+fOio+P1+7du/Xggw+qa9euOnHiRK7LGTNmjHr16qWff/5ZnTt31hNPPKFLly4VqKadO3eqV69e6tOnj/bs2aPRo0dr5MiRWrBggSTpxx9/1HPPPaexY8fq4MGD2rBhg9q3by/pz29B+/btq6eeekr79+/Xli1b9Mgjj4jH2MMRFi9erFGjRmn8+PHav3+/JkyYoJEjR2rhwoWSpHfffVfr1q3Tv//9bx08eFCLFy+2DfY//PCDJGn+/Pk6e/asbfpW/fOf/9SUKVP09ttv6+eff1ZERIQefvhh/fbbb3nWsGrVKr3zzjv64IMP9Ntvv2nt2rVq0qTJ7W0UoJRy5Hg7f/58denSRV5eXvr73/9uFwAkaebMmRoyZIgGDhyoPXv2aN26dapdu7akP7+4e+ihh/TNN9/oo48+0r59+zRx4kQ5Ozvf0vrHx8fr4MGD2rhxoy2w37hxQ+PGjdNPP/2ktWvX6tixY3bh4vTp02rfvr2sVqs2b96snTt36qmnntIff/yh9u3bq1atWlq0aJGt/40bN7R48WI99dRTt1QbSr6iHG/nzp2rv//97/Ly8tJDDz1k+/v0psjISC1dulTvvvuu9u/frw8++MB2FD63n+lb8fbbbyskJES7d++2heLy5ctrwYIF2rdvn/75z39q9uzZeuedd2zzfPbZZ+rRo4c6d+6s3bt3Kz4+Xq1bt5Yk29/N/7vuu3fv1s8//6zo6Ohbqu2OZwC5mD9/vuHl5WWb/uqrrwxJxtq1a/Oct1GjRsb06dNt04GBgcY777xjm5ZkvPHGG7bpq1evGpKMzz//PMdldujQwXj++eezfe/xxx83OnXqZNf2j3/8w2jYsKFhGIaxatUqw9PT00hJScky786dOw1JxrFjx/JcL6Cw/XU/Cw4ONpYsWWLXZ9y4cUabNm0MwzCMYcOGGffdd5+RmZmZ7fIkGWvWrMnzc3Pr5+/vb4wfP96urVWrVsbgwYPzrGHKlClG3bp1jfT09DxrAPCn4jbeZmRkGAEBAbbPP3/+vOHq6mocOXLE1sff398YMWJEtvN/8cUXhpOTk3Hw4MFs34+NjTVCQkLs2t555x0jMDDQNh0VFWX4+PgYaWlpOdZpGIbxww8/GJKMK1euGIZhGMOHDzdq1qyZ4++gt956y2jQoIFtetWqVUa5cuWMq1ev5vo5KPkcNd7++uuvRpkyZYzz588bhmEYa9asMWrWrGlb7sGDBw1JxsaNG7OdP6+f6ez+Pu7WrZsRFRVlmw4MDDS6d++eZ62TJ082WrRoYZtu06aN8cQTT+TY/6GHHjIGDRpkmx42bJjRsWPHPD+ntOFINwqkZcuWdtNXr17Vyy+/rAYNGqhChQoqV66c9u/fn+c3702bNrX928PDQ56enllOEcuv/fv3q23btnZtbdu21W+//aaMjAx16tRJgYGBqlWrlp588kktXrxY165dkySFhITo/vvvV5MmTfTYY49p9uzZ+v333wtUB3A7UlNTdfjwYfXv31/lypWzvd58803baWT9+vVTQkKC6tWrp+eee05ffvllodaQkpKiM2fOZLs/7d+/P88aHnvsMV2/fl21atXSgAEDtGbNmlv+Nh7Anxw13m7cuFGpqanq3LmzJMnb21udOnXSvHnzJEnnzp3TmTNndP/992c7f0JCgqpXr666devmaz1z0qRJkyzXne7cuVNdu3ZVjRo1VL58eXXo0EGSbNsgISFB7dq1U5kyZbJdZr9+/XTo0CF99913kqQFCxaoV69e8vDwuK1aUbIU5Xg7b948RUREyNvbW5LUuXNnJScna/PmzZL+/Jl1dna2/Sz/VV4/0/n1198n0p/Xubdt21a+vr4qV66c3njjDbvfJwkJCTnu55I0YMAALV26VP/5z3+Unp6uJUuWcNZINgjdKJC/Dkwvv/yy1qxZowkTJujrr79WQkKCmjRpovT09FyX89dfHhaLRZmZmYVer/Tn6TO7du3S0qVL5efnp1GjRikkJESXL1+Ws7OzNm7cqM8//1wNGzbU9OnTVa9ePR09etSUWoCcXL16VZI0e/ZsJSQk2F579+61/YHYvHlzHT16VOPGjdP169fVq1cvu/sTFIXcaggICNDBgwf1/vvvy93dXYMHD1b79u2zXCMGIG+OGm/nzp2rS5cuyd3dXS4uLnJxcdH69eu1cOFCZWZmyt3dPdfPy+t9JyenLJdwZfc74q/rn5qaqoiICHl6emrx4sX64YcftGbNGkmybYO8Prtq1arq2rWr5s+fr6SkJH3++eeEhFKoqMbbjIwMLVy4UJ999pltXypbtqwuXbpk+xLLUfvT9u3b9cQTT6hz58769NNPtXv3bo0YMcLu90len921a1dZrVatWbNGn3zyiW7cuFHkf5OUBNy9HIXim2++Ub9+/Ww3Vrh69aqOHTtWpDU0aNBA33zzTZa66tata7uGzMXFReHh4QoPD1dsbKwqVKigzZs365FHHpHFYlHbtm3Vtm1bjRo1SoGBgVqzZo1iYmKKdD1Quvn4+Mjf319HjhzRE088kWM/T09P9e7dW71791bPnj314IMP6tKlS6pUqZLKlCmjjIyMAtfg6ekpf39/ffPNN3bfun/zzTe267jyqsHd3V1du3ZV165dNWTIENWvX1979uxR8+bNC1wXgKIZby9evKiPP/5Yy5YtU6NGjWztGRkZuueee/Tll1/qwQcfVFBQkOLj43XvvfdmWUbTpk116tQp/frrr9ke7a5SpYoSExNlGIYsFoukP4+o5eXAgQO6ePGiJk6cqICAAEl/3rPlr5+9cOFC3bhxI8cjg08//bT69u2r6tWrKzg4OMuZPbjzFdV4u379el25ckW7d++2u6fB3r17FR0drcuXL6tJkybKzMzU//3f/yk8PDzLMvL6ma5SpYrdXdozMjK0d+/ebPfN//Xtt98qMDBQI0aMsLUdP348y2fHx8fneI22i4uLoqKiNH/+fLm6uqpPnz55BvXSiNCNQlGnTh2tXr1aXbt2lcVi0ciRI007Yn3+/PksA7Ofn59eeukltWrVSuPGjVPv3r21fft2vffee7a7UH766ac6cuSI2rdvr4oVK2r9+vXKzMxUvXr19P333ys+Pl4PPPCAqlatqu+//17nz59XgwYNTFkHIDdjxozRc889Jy8vLz344INKS0vTjz/+qN9//10xMTGaOnWq/Pz81KxZMzk5OWnFihXy9fVVhQoVJMn2h3Dbtm1ltVpVsWLFHD/r6NGjWfanOnXq6B//+IdiY2MVHBys0NBQzZ8/XwkJCVq8eLEk5VrDggULlJGRobCwMJUtW1YfffSR3N3dFRgYaNYmA0qNohhvFy1apMqVK6tXr162QHxT586dNXfuXD344IMaPXq0nn32WVWtWlUPPfSQrly5om+++UbDhg1Thw4d1L59ez366KOaOnWqateurQMHDshisejBBx9Ux44ddf78eU2aNEk9e/bUhg0b9Pnnn8vT0zPX2mrUqCFXV1dNnz5dzz77rPbu3atx48bZ9Rk6dKimT5+uPn36aPjw4fLy8tJ3332n1q1b2+6AfvNo+ZtvvqmxY8cW6vZDyVEU4+3cuXPVpUsXhYSE2LU3bNhQL774ohYvXqwhQ4YoKipKTz31lN59912FhITo+PHjOnfunHr16pXnz/R9992nmJgYffbZZwoODtbUqVN1+fLlPNe/Tp06OnHihJYtW6ZWrVrps88+s505clNsbKzuv/9+BQcHq0+fPvrjjz+0fv16vfrqq7Y+Tz/9tO1v5r8eAMP/5+BrylHM5XRjl99//92u39GjR417773XcHd3NwICAoz33nsvy00dsruxy19vPuHl5WXMnz8/x3o6dOhgSMryGjdunGEYhrFy5UqjYcOGRpkyZYwaNWoYkydPts379ddfGx06dDAqVqxouLu7G02bNjWWL19uGIZh7Nu3z4iIiDCqVKliWK1Wo27dunY3pQHM9Nf9zDAMY/HixUZoaKjh6upqVKxY0Wjfvr2xevVqwzAM48MPPzRCQ0MNDw8Pw9PT07j//vuNXbt22eZdt26dUbt2bcPFxcXupkR/ld2+JMn4+uuvjYyMDGP06NFGtWrVjDJlyhghISF2N13KrYY1a9YYYWFhhqenp+Hh4WHcddddxqZNmwpvgwF3oOI03jZp0sR208S/Wr58ueHq6mq7IdSsWbOMevXqGWXKlDH8/PyMYcOG2fpevHjRiI6ONipXrmy4ubkZjRs3Nj799FPb+zNnzjQCAgIMDw8PIzIy0hg/fnyWG6l169YtSw1LliwxgoKCDKvVarRp08ZYt26dIcnYvXu3rc9PP/1kPPDAA0bZsmWN8uXLG+3atTMOHz5st5yRI0cazs7OxpkzZ7JdV9x5inq8TUxMNFxcXIx///vf2dYzaNAgo1mzZoZhGMb169eNF1980fDz8zNcXV2N2rVrG/PmzbP1ze1nOj093Rg0aJBRqVIlo2rVqkZcXFy2N1L7398LN/3jH/8wKleubJQrV87o3bu38c4772TZRqtWrbJtI29vb+ORRx7Jspx27doZjRo1ynY9YRgWw+CZSAAAAChd+vfvr/Pnz+frmeUAcmYYhurUqaPBgwdzWWYOOL0cAAAApUZycrL27NmjJUuWELiB23T+/HktW7ZMiYmJPJs7F4RuAAAAlBrdunXTjh079Oyzz6pTp06OLgco0apWrSpvb299+OGHud5DprTj9HIAAAAAAEzCc7oBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMMn/A5uA6bQqN+4DAAAAAElFTkSuQmCC\n" + }, + "metadata": {} + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "## Untrained model vs Trained model\n", + "Comparision for the sake of fine-tuning (checking if fine-tuning the model acctually helped)" + ], + "metadata": { + "id": "Nab5w3oWaTeu" + }, + "id": "Nab5w3oWaTeu" + }, + { + "cell_type": "code", + "source": [ + "### Creating the model\n", + "from transformers import BertConfig, BertForSequenceClassification\n", + "config = BertConfig.from_pretrained(\"bert-base-uncased\", num_labels=num_classes)\n", + "untrained_model = BertForSequenceClassification(config)\n", + "untrained_model.to(device)" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "0zmAoQ88YuFk", + "outputId": "f146417a-5e41-4b54-ff8c-0344898e88c6" + }, + "id": "0zmAoQ88YuFk", + "execution_count": 41, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "BertForSequenceClassification(\n", + " (bert): BertModel(\n", + " (embeddings): BertEmbeddings(\n", + " (word_embeddings): Embedding(30522, 768, padding_idx=0)\n", + " (position_embeddings): Embedding(512, 768)\n", + " (token_type_embeddings): Embedding(2, 768)\n", + " (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " )\n", + " (encoder): BertEncoder(\n", + " (layer): ModuleList(\n", + " (0-11): 12 x BertLayer(\n", + " (attention): BertAttention(\n", + " (self): BertSdpaSelfAttention(\n", + " (query): Linear(in_features=768, out_features=768, bias=True)\n", + " (key): Linear(in_features=768, out_features=768, bias=True)\n", + " (value): Linear(in_features=768, out_features=768, bias=True)\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " )\n", + " (output): BertSelfOutput(\n", + " (dense): Linear(in_features=768, out_features=768, bias=True)\n", + " (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " )\n", + " )\n", + " (intermediate): BertIntermediate(\n", + " (dense): Linear(in_features=768, out_features=3072, bias=True)\n", + " (intermediate_act_fn): GELUActivation()\n", + " )\n", + " (output): BertOutput(\n", + " (dense): Linear(in_features=3072, out_features=768, bias=True)\n", + " (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " )\n", + " )\n", + " )\n", + " )\n", + " (pooler): BertPooler(\n", + " (dense): Linear(in_features=768, out_features=768, bias=True)\n", + " (activation): Tanh()\n", + " )\n", + " )\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " (classifier): Linear(in_features=768, out_features=5, bias=True)\n", + ")" + ] + }, + "metadata": {}, + "execution_count": 41 + } + ] + }, + { + "cell_type": "code", + "source": [ + "def evaluate_model(model,\n", + " dataloader,\n", + " loss_fn,\n", + " device):\n", + " model.eval()\n", + " total_loss = 0\n", + " correct_predictions = 0\n", + " total_samples = 0\n", + "\n", + " with torch.no_grad():\n", + " for batch in dataloader:\n", + " input_ids = batch['input_ids'].to(device)\n", + " attention_mask = batch['attention_mask'].to(device)\n", + " labels = batch['label'].to(device)\n", + " outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)\n", + " loss = outputs.loss\n", + " logits = outputs.logits\n", + " total_loss += loss.item()\n", + " predictions = torch.argmax(logits, dim=1)\n", + " correct_predictions += (predictions == labels).sum().item()\n", + " total_samples += labels.size(0)\n", + "\n", + " avg_loss = total_loss / len(dataloader)\n", + " accuracy = correct_predictions / total_samples\n", + " return avg_loss, accuracy\n", + "\n", + "# Eval both the models\n", + "trained_test_loss, trained_test_accuracy = results['test_loss'][0], results['test_accuracy'][0] # Eval already done\n", + "untrained_test_loss, untrained_test_accuracy = evaluate_model(\n", + " untrained_model,\n", + " test_dataloader,\n", + " loss_fn,\n", + " device\n", + ")" + ], + "metadata": { + "id": "7cqIjc1qaoH2" + }, + "id": "7cqIjc1qaoH2", + "execution_count": 42, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "### Plots to compare the models\n", + "loss_metrics = ['Trained Model', 'Untrained Model']\n", + "loss_values = [trained_test_loss, untrained_test_loss]\n", + "accuracy_metrics = ['Trained Model', 'Untrained Model']\n", + "accuracy_values = [trained_test_accuracy, untrained_test_accuracy]\n", + "\n", + "# Loss\n", + "plt.figure(figsize=(10, 5))\n", + "plt.subplot(1, 2, 1)\n", + "plt.bar(loss_metrics, loss_values, color=['blue', 'orange'])\n", + "plt.ylabel('Loss')\n", + "plt.title('Test Loss (lower = better)')\n", + "\n", + "# Accuracy\n", + "plt.subplot(1, 2, 2)\n", + "plt.bar(accuracy_metrics, accuracy_values, color=['blue', 'orange'])\n", + "plt.ylim(0, 1)\n", + "plt.ylabel('Accuracy')\n", + "plt.title('Test Accuracy (higher = better)')\n", + "\n", + "plt.tight_layout()\n", + "plt.show()\n", + "plt.savefig(root_dir + \"/model_comparision.png\")" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 524 + }, + "id": "IQzNS6Uua8i1", + "outputId": "245e892f-317e-44db-aac5-c909c0b3b9a9" + }, + "id": "IQzNS6Uua8i1", + "execution_count": 46, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA94AAAHqCAYAAADyGZa5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABbHElEQVR4nO3deVxUdf///+cgMqAILggIorjvomGQmVuhqF2kmWnSJwi3Fk2TFjXLpUXK1Kwr07Rc6uuWllZXpilprmlqZIu5G6aCkpegWKBwfn/4c64mFhE5DMvjfrud2815z/uc8zojM+95ztkshmEYAgAAAAAApnBydAEAAAAAAJRlBG8AAAAAAExE8AYAAAAAwEQEbwAAAAAATETwBgAAAADARARvAAAAAABMRPAGAAAAAMBEBG8AAAAAAExE8AYAAAAAwEQEb6CcmDp1qpo2bars7Gxbm8Vi0aRJkxxXVAm0adMmWSwWrVy50tGl3LS1a9fK3d1dZ8+edXQpAIBismvXLrm4uOi3336ztQUGBupf//rXdee9NgZu2rTphtdblsZPqWxtD98HSgaCN0o0i8VSoKkwA8Q/Xbp0SZMmTSrwskrTB3JaWppee+01jRkzRk5OvO2L2/bt2zVp0iSdP38+x3NTpkzR6tWrTVlvjx491LBhQ8XFxZmyfAClS0keU/9uzZo1slgs8vPzs/uxGAUzfvx4DRw4UHXr1nV0KfgHvg+Ub86OLgDIz4cffmj3+IMPPtD69etztDdr1uym13Xp0iVNnjxZktSlS5ebXl5JMn/+fF25ckUDBw50dCnl0vbt2zV58mQ9/PDDqlq1qt1zU6ZMUb9+/dSnTx9T1v3II4/o6aef1uTJk1WlShVT1gGgdCgtY+rixYsVGBio48eP6+uvv1ZYWNhN11NeJCQkaMOGDdq+fXuh5u/UqZP+/PNPubi4FHFlkPg+UN4RvFGi/d///Z/d42+//Vbr16/P0Y78LViwQPfcc49cXV0dXUqhpaenq3Llyo4uo1T466+/5OLiIicnJ91333164okntGLFCg0aNMjRpQFwoNIwpqanp+vTTz9VXFycFixYoMWLF5fY4F0Sx6UFCxaoTp06uu222wo1v5OTU4n/rlASX/eSiu8DJQvHnKLUy87O1syZM9WiRQu5urrKx8dHjzzyiP773//a9du9e7fCw8Pl5eUlNzc31atXz/bBc/z4cdWsWVOSNHnyZNvhdkVx/vPRo0d1//33q3r16qpUqZJuu+02ffHFFzn6/fvf/1aLFi1UqVIlVatWTe3atdOSJUtsz1+4cEFPPvmkAgMDZbVa5e3trW7dumnv3r35rv/YsWPat29fgb+4fP/99+rZs6c8PDzk7u6uu+66S99++63t+fPnz6tChQp66623bG0pKSlycnJSjRo1ZBiGrf2xxx6Tr6+v3fJ37typHj16yNPTU5UqVVLnzp21bds2uz6TJk2SxWLRL7/8osjISFWrVk133HFHgeovKllZWXruuefk6+urypUr65577tGJEydy9Lve9kyaNEnPPPOMJKlevXq2v63jx4/LYrEoPT1dixYtsrU//PDDtnlPnjypQYMGycfHR1arVS1atND8+fPt1n/tlIdly5bp+eefl7+/vypVqqS0tDRJkre3t1q3bq1PP/3UhFcJQFnj6DF11apV+vPPP3X//ffrgQce0CeffKK//vorR7+//vpLkyZNUuPGjeXq6qpatWqpb9++OnLkiN22vPnmm2rVqpVcXV1Vs2ZN9ejRQ7t377bVabFYtHDhwhzL/2e9+Y1L+/bt08MPP6z69evL1dVVvr6+GjRokP74448cyz158qQGDx4sPz8/Wa1W1atXT4899pgyMzN19OhRWSwWvfHGGznm2759uywWi5YuXZrv67d69WrdeeedslgsuT6/detWhYSEyNXVVfXr19cHH3xg93xe53jPmjVL9evXl5ubm0JCQrRlyxZ16dIl16MZsrOz9corr6h27dpydXXVXXfdpcOHD+fox/cBvg+UN+zxRqn3yCOPaOHChYqJidHIkSN17Ngxvf322/r++++1bds2VaxYUWfOnFH37t1Vs2ZNjR07VlWrVtXx48f1ySefSJJq1qyp2bNn67HHHtO9996rvn37SpJat259U7UlJyfr9ttv16VLlzRy5EjVqFFDixYt0j333KOVK1fq3nvvlSTNmzdPI0eOVL9+/TRq1Cj99ddf2rdvn3bu3KnIyEhJ0qOPPqqVK1dqxIgRat68uf744w9t3bpV+/fv1y233JJnDdcON8uvzzU///yzOnbsKA8PDz377LOqWLGi3n33XXXp0kXffPONQkNDVbVqVbVs2VKbN2/WyJEjJV0dyC0Wi86dO6dffvlFLVq0kCRt2bJFHTt2tC3/66+/Vs+ePRUcHKyJEyfKyclJCxYs0J133qktW7YoJCTErp77779fjRo10pQpU+wC/T9lZ2fr3Llz190+SfL09FTFihWv2++VV16RxWLRmDFjdObMGc2cOVNhYWFKSEiQm5tbgbenb9++OnjwoJYuXao33nhDXl5ekq7+zX344YcaMmSIQkJCNGzYMElSgwYNJF3927nttttksVg0YsQI1axZU19++aUGDx6stLQ0Pfnkk3b1vvTSS3JxcdHTTz+tjIwMu8MEg4ODTTtvDEDZ4ugxdfHixeratat8fX31wAMPaOzYsfr88891//332/pkZWXpX//6l+Lj4/XAAw9o1KhRunDhgtavX6+ffvrJ9jk6ePBgLVy4UD179tSQIUN05coVbdmyRd9++63atWtXqNcnt3Fp/fr1Onr0qGJiYuTr66uff/5Zc+fO1c8//6xvv/3WFoJPnTqlkJAQnT9/XsOGDVPTpk118uRJrVy5UpcuXVL9+vXVoUMHLV68WKNHj87xulSpUkW9e/fOs7aTJ08qMTExz/H+8OHD6tevnwYPHqzo6GjNnz9fDz/8sIKDg23jdm5mz56tESNGqGPHjho9erSOHz+uPn36qFq1aqpdu3aO/q+++qqcnJz09NNPKzU1VVOnTtWDDz6onTt32vrwfYDvA+WSAZQiw4cPN/7+Z7tlyxZDkrF48WK7fmvXrrVrX7VqlSHJ+O677/Jc9tmzZw1JxsSJEwtUy8aNGw1JxooVK/Ls8+STTxqSjC1bttjaLly4YNSrV88IDAw0srKyDMMwjN69exstWrTId32enp7G8OHDC1Tb3z3//POGJOPChQs5nvvn9vbp08dwcXExjhw5Yms7deqUUaVKFaNTp062tuHDhxs+Pj62x7GxsUanTp0Mb29vY/bs2YZhGMYff/xhWCwW48033zQMwzCys7ONRo0aGeHh4UZ2drZt3kuXLhn16tUzunXrZmubOHGiIckYOHBggbbx2LFjhqQCTRs3bsx3Wdf+X/39/Y20tDRb+0cffWRIKtT2vP7664Yk49ixYznWV7lyZSM6OjpH++DBg41atWoZKSkpdu0PPPCA4enpaVy6dMmu3vr169va/mnKlCmGJCM5OTnfbQdQvpSkMdUwDCM5OdlwdnY25s2bZ2u7/fbbjd69e9v1mz9/viHJmDFjRo5lXPs8/vrrrw1JxsiRI/Psc23sWLBgQY4+/6w9v3Ept8/epUuXGpKMzZs329qioqIMJyenXF+3azW9++67hiRj//79tucyMzMNLy+vXMeKv9uwYYMhyfj8889zPFe3bt0c9Zw5c8awWq3GU089ZWu7NqZcGyszMjKMGjVqGLfeeqtx+fJlW7+FCxcakozOnTvnmLdZs2ZGRkaGrf3NN980JBk//vijbVv5PnAsx/r4PlD2cag5SrUVK1bI09NT3bp1U0pKim0KDg6Wu7u7Nm7cKEm2C1j85z//0eXLl4utvjVr1igkJMTusCh3d3cNGzZMx48f1y+//GKr7/fff9d3332X57KqVq2qnTt36tSpUzdUwx9//CFnZ2e5u7vn2y8rK0tfffWV+vTpo/r169vaa9WqpcjISG3dutV2uFLHjh2VnJysAwcOSLq6Z7tTp07q2LGjtmzZIunqXnDDMGx7vBMSEnTo0CFFRkbqjz/+sP1fpaen66677tLmzZtzXL320UcfLdA2+vr6av369QWagoKCCrTMqKgou4uP9OvXT7Vq1dKaNWsKvT0FZRiGPv74Y0VERMgwDLu/7fDwcKWmpuY4xSA6Otr2y/s/VatWTdLVUwIAIC+OHlOXLVtmOxf1moEDB+rLL7+0O9T9448/lpeXl5544okcy7i2d/njjz+WxWLRxIkT8+xTGLmNS3//7P3rr7+UkpJiO8f62md1dna2Vq9erYiIiFz3tl+rqX///nJ1ddXixYttz61bt04pKSnXPRf/2qHt1z7z/6l58+Z2R6HVrFlTTZo00dGjR/Nc5u7du/XHH39o6NChcnb+34GyDz74YJ7riYmJsdvLem2d19bD94GC4/tA2cKh5ijVDh06pNTUVHl7e+f6/JkzZyRJnTt31n333afJkyfrjTfeUJcuXdSnTx9FRkbKarWaVt9vv/2m0NDQHO3Xrhj722+/qWXLlhozZow2bNigkJAQNWzYUN27d1dkZKQ6dOhgm2fq1KmKjo5WQECAgoOD1atXL0VFRdmF5Jtx9uxZXbp0SU2aNMm13uzsbJ04cUItWrSwDaJbtmxR7dq19f333+vll19WzZo1NW3aNNtzHh4etoHt0KFDkq4OCHlJTU21G8jr1atXoNpdXV2L/OI7jRo1sntssVjUsGFDHT9+XFLhtqegzp49q/Pnz2vu3LmaO3durn2u/W1fk99rZfz/h+XdzJdNAGWfo8fU//f//p9CQkL0xx9/2EJk27ZtlZmZqRUrVtgOwT1y5IiaNGliFwT/6ciRI/Lz81P16tULXU9ucvusPXfunCZPnqxly5bl+GxOTU2VdPVzPS0tTS1btsx3+VWrVlVERISWLFmil156SdLVw8z9/f115513FqhGI49DsevUqZOjrVq1ajnO3/+7a/cCb9iwoV27s7OzAgMDC7Sea+PgtfXwfaDg+D5QthC8UaplZ2fL29vb7pfhv7t2cZdr99v+9ttv9fnnn2vdunUaNGiQpk+frm+//fa6e4PN1qxZMx04cED/+c9/tHbtWn388cd65513NGHCBNvtWPr376+OHTtq1apV+uqrr/T666/rtdde0yeffKKePXvmuewaNWroypUrunDhQpHdPsLPz0/16tXT5s2bFRgYKMMw1L59e9WsWVOjRo3Sb7/9pi1btuj222+33Tf82q+9r7/+utq0aZPrcv/5/5DXL7b/lJWVpbNnzxaob/Xq1YvkNimF2Z4bXfb//d//5TmQ//Ncyfxeq2tfdq6dTwYAuXHkmHro0CHbUV//DDrS1fB5LXgXlbzCR1ZWVp7z5PZZ279/f23fvl3PPPOM2rRpI3d3d2VnZ6tHjx6F2tMZFRWlFStWaPv27WrVqpU+++wzPf7447bxNC81atSQpDyDdIUKFXJtzyuoF9b11sP3gRtfNt8HygaCN0q1Bg0aaMOGDerQoUOBPpRvu+023XbbbXrllVe0ZMkSPfjgg1q2bJmGDBliyq9/devWtR2O/Xe//vqr7flrKleurAEDBmjAgAHKzMxU37599corr2jcuHG2W3vUqlVLjz/+uB5//HGdOXNGt9xyi1555ZV8g3fTpk0lXb26eX4XtqlZs6YqVaqUZ71OTk4KCAiwtXXs2FGbN29WvXr11KZNG1WpUkVBQUHy9PTU2rVrtXfvXtuPBtL/LhLi4eFR5L9GnzhxosC/hm/cuLFA95S99gv2NYZh6PDhw7bX8Ea2J7+/rdyeq1mzpqpUqaKsrKwiea2OHTsmLy8v25dmAMiNI8fUxYsXq2LFivrwww9zBLetW7fqrbfeUmJiourUqaMGDRpo586dunz5cp4Xx2rQoIHWrVunc+fO5bnX+9oeyPPnz9u1X9vLWxD//e9/FR8fr8mTJ2vChAm29n+OITVr1pSHh4d++umn6y6zR48eqlmzphYvXqzQ0FBdunRJDz300HXn+/t4X1SufU85fPiwunbtamu/cuWKjh8/XqiL0PJ9oODP8X2gbOEcb5Rq/fv3V1ZWlu1wrL+7cuWKbTD973//m+MX3Wu/SmZkZEiSKlWqJCnnAHwzevXqpV27dmnHjh22tvT0dM2dO1eBgYFq3ry5JOW45YiLi4uaN28uwzB0+fJlZWVl2Q5Xu8bb21t+fn62+vPSvn17SbLdPiUvFSpUUPfu3fXpp5/aDp+Srl5Nc8mSJbrjjjvk4eFha+/YsaOOHz+u5cuX2w49d3Jy0u23364ZM2bo8uXLdueSBQcHq0GDBpo2bZouXryYY/0F/YU6N2ac0/XBBx/owoULtscrV67U6dOnbT9y3Mj2XLvfaG5/W5UrV87RXqFCBd133336+OOPc/2SdqOv1Z49e2x/BwCQF0eOqYsXL1bHjh01YMAA9evXz266dguma7fSuu+++5SSkqK33347x3Ku1XXffffJMAy7H4D/2cfDw0NeXl7avHmz3fPvvPNOgWqW/rd395+vx8yZM+0eOzk5qU+fPvr8889zHY//Pr+zs7MGDhyojz76SAsXLlSrVq0KFHD9/f0VEBBw3fH+RrRr1041atTQvHnzdOXKFVv74sWL8z1EPT98H+D7QHnFHm+Uap07d9YjjzyiuLg4JSQkqHv37qpYsaIOHTqkFStW6M0331S/fv20aNEivfPOO7r33nvVoEEDXbhwQfPmzZOHh4d69eol6eqhOc2bN9fy5cvVuHFjVa9eXS1btrzu+Vgff/yxbQ/230VHR2vs2LFaunSpevbsqZEjR6p69epatGiRjh07po8//th22Fj37t3l6+urDh06yMfHR/v379fbb7+tu+++W1WqVNH58+dVu3Zt9evXT0FBQXJ3d9eGDRv03Xffafr06fnWV79+fbVs2VIbNmyw3WM1Ly+//LLWr1+vO+64Q48//ricnZ317rvvKiMjQ1OnTrXrey1UHzhwQFOmTLG1d+rUSV9++aWsVqtuvfVWW7uTk5Pee+899ezZUy1atFBMTIz8/f118uRJbdy4UR4eHvr888/zrS8vZpzTVb16dd1xxx2KiYlRcnKyZs6cqYYNG2ro0KGSbmx7goODJUnjx4/XAw88oIoVKyoiIkKVK1dWcHCwNmzYoBkzZtgO4Q8NDdWrr76qjRs3KjQ0VEOHDlXz5s117tw57d27Vxs2bCjw7VLOnDmjffv2afjw4UX6+gAoexw1pu7cuVOHDx/WiBEjcq3L399ft9xyixYvXqwxY8YoKipKH3zwgWJjY7Vr1y517NhR6enp2rBhgx5//HH17t1bXbt21UMPPaS33npLhw4dsh32vWXLFnXt2tW2riFDhujVV1/VkCFD1K5dO23evFkHDx4s8Gvm4eGhTp06aerUqbp8+bL8/f311Vdf5brXecqUKfrqq6/UuXNnDRs2TM2aNdPp06e1YsUKbd261XbROunq4eZvvfWWNm7cqNdee63A9fTu3VurVq2SYRhFciSfi4uLJk2apCeeeEJ33nmn+vfvr+PHj2vhwoVq0KBBodbB9wG+D5RbxXkJdeBm/fPWJ9fMnTvXCA4ONtzc3IwqVaoYrVq1Mp599lnj1KlThmEYxt69e42BAwcaderUMaxWq+Ht7W3861//Mnbv3m23nO3btxvBwcGGi4vLdW+Dcu22DXlN124hduTIEaNfv35G1apVDVdXVyMkJMT4z3/+Y7esd9991+jUqZNRo0YNw2q1Gg0aNDCeeeYZIzU11TCMq7fzeOaZZ4ygoCCjSpUqRuXKlY2goCDjnXfeKdDrNmPGDMPd3T3H7SVy28a9e/ca4eHhhru7u1GpUiWja9euxvbt23Ndrre3d47bUmzdutWQZHTs2DHXeb7//nujb9++tm2tW7eu0b9/fyM+Pt7W59rtQ86ePVug7StK1/5fly5daowbN87w9vY23NzcjLvvvtv47bffcvQvyPYYhmG89NJLhr+/v+Hk5GR3K5Fff/3V6NSpk+Hm5mZIsruVSHJysjF8+HAjICDAqFixouHr62vcddddxty5c3PUm9dt7WbPnm1UqlTJ7lYoAGAYJWdMfeKJJwxJdrey/KdJkyYZkowffvjBMIyrt2oaP368Ua9ePdvnY79+/eyWceXKFeP11183mjZtari4uBg1a9Y0evbsaezZs8fW59KlS8bgwYMNT09Po0qVKkb//v2NM2fO5Hk7sdzGpd9//9249957japVqxqenp7G/fffb5w6dSrXbf7tt9+MqKgoo2bNmobVajXq169vDB8+3O72W9e0aNHCcHJyMn7//fc8X5d/2rt3b47bmBrG1duJ3X333Tn6d+7cOddbgv3zVltvvfWWUbduXcNqtRohISHGtm3bjODgYKNHjx455v3neJTXbdv4PsD3gfLGYhhFfEUFACVOamqq6tevr6lTp2rw4MGOLgfFqG3bturSpYveeOMNR5cCALgBbdu2VfXq1RUfH39D8911113y8/PThx9+aFJlVy/6VbNmTfXt21fz5s0zbT0oOnwfcDzO8QbKAU9PTz377LN6/fXXC30vSZQ+a9eu1aFDhzRu3DhHlwIAuAG7d+9WQkKCoqKibnjeKVOmaPny5Td0kbj8/PXXXznOYf/ggw907ty5Al2cDI7H94GSgT3eAAAAQAnw008/ac+ePZo+fbpSUlJ09OhR251NHGXTpk0aPXq07r//ftWoUUN79+7V+++/r2bNmmnPnj1FcksuoDzg4moAAABACbBy5Uq9+OKLatKkiZYuXerw0C1JgYGBCggI0FtvvWW7PVtUVJReffVVQjdwA9jjDQAArmvz5s16/fXXtWfPHp0+fVqrVq1Snz598p1n06ZNio2N1c8//6yAgAA9//zzevjhh4ulXgAAShLO8QYAANeVnp6uoKAgzZo1q0D9jx07prvvvltdu3ZVQkKCnnzySQ0ZMkTr1q0zuVIAAEoe9ngDAIAbYrFYrrvHe8yYMfriiy/0008/2doeeOABnT9/XmvXri2GKgEAKDnK3Tne2dnZOnXqlKpUqSKLxeLocgAAyMEwDF24cEF+fn5yciqdB6ft2LFDYWFhdm3h4eF68skn85wnIyNDGRkZtsfZ2dk6d+6catSowZgNAChxbmS8LnfB+9SpUwoICHB0GQAAXNeJEydUu3ZtR5dRKElJSfLx8bFr8/HxUVpamv7880+5ubnlmCcuLk6TJ08urhIBACgSBRmvy13wrlKliqSrL46Hh4eDqwEAIKe0tDQFBATYxqzyYty4cYqNjbU9Tk1NVZ06dRizAQAl0o2M1+UueF87VM3Dw4NBHABQopXmw6t9fX2VnJxs15acnCwPD49c93ZLktVqldVqzdHOmA0AKMkKMl6XzhPHAABAida+fXvFx8fbta1fv17t27d3UEUAADgOwRsAAFzXxYsXlZCQoISEBElXbxeWkJCgxMRESVcPE4+KirL1f/TRR3X06FE9++yz+vXXX/XOO+/oo48+0ujRox1RPgAADkXwBgAA17V79261bdtWbdu2lSTFxsaqbdu2mjBhgiTp9OnTthAuSfXq1dMXX3yh9evXKygoSNOnT9d7772n8PBwh9QPAIAjlbv7eKelpcnT01OpqamcLwYAKJEYq67idQAAlGQ3Mk6xxxsAAAAAABMRvAEAAAAAMBHBGwAAAAAAExG8AQAAAAAwEcEbAAAAAAATOTR4b968WREREfLz85PFYtHq1auvO09GRobGjx+vunXrymq1KjAwUPPnzze/WAAAAAAACsHZkStPT09XUFCQBg0apL59+xZonv79+ys5OVnvv/++GjZsqNOnTys7O9vkSgEAAAAAKByHBu+ePXuqZ8+eBe6/du1affPNNzp69KiqV68uSQoMDDSpOgAAAAAAbl6pOsf7s88+U7t27TR16lT5+/urcePGevrpp/Xnn3/mOU9GRobS0tLsJgAAAAAAiotD93jfqKNHj2rr1q1ydXXVqlWrlJKSoscff1x//PGHFixYkOs8cXFxmjx5cjFXCgAAAADAVaVqj3d2drYsFosWL16skJAQ9erVSzNmzNCiRYvy3Os9btw4paam2qYTJ04Uc9UAAAAAgPKsVO3xrlWrlvz9/eXp6Wlra9asmQzD0O+//65GjRrlmMdqtcpqtRZnmQAAAAAA2JSqPd4dOnTQqVOndPHiRVvbwYMH5eTkpNq1azuwMgAAAAAAcufQ4H3x4kUlJCQoISFBknTs2DElJCQoMTFR0tXDxKOiomz9IyMjVaNGDcXExOiXX37R5s2b9cwzz2jQoEFyc3NzxCYAAAAAAJAvhx5qvnv3bnXt2tX2ODY2VpIUHR2thQsX6vTp07YQLknu7u5av369nnjiCbVr1041atRQ//799fLLLxd77QCKwRKLoysACi7ScHQFAACghHJo8O7SpYsMI+8vKgsXLszR1rRpU61fv97EqgAAAAAAKDql6hxvAAAAAABKG4I3AAAAAAAmIngDAAAAAGAigjcAAAAAACYieAMAAAAAYCKCNwAAAAAAJiJ4AwAAAABgIoI3AAAAAAAmIngDAAAAAGAigjcAAAAAACYieAMAAAAAYCKCNwAAAAAAJiJ4AwAAAABgIoI3AAAAAAAmIngDAAAAAGAigjcAAAAAACYieAMAAAAAYCKCNwAAAAAAJiJ4AwAAAABgIoI3AAAAAAAmIngDAAAAAGAigjcAAAAAACYieAMAAAAAYCKCNwAAAAAAJiJ4AwAAAABgIoI3AAAAAAAmIngDAAAAAGAigjcAAAAAACYieAMAAAAAYCKCNwAAAAAAJiJ4AwAAAABgIoI3AAAAAAAmIngDAAAAAGAigjcAAAAAACYieAMAAAAAYCKCNwAAAAAAJiJ4AwAAAABgIoI3AAAAAAAmIngDAAAAAGAigjcAAAAAACZyaPDevHmzIiIi5OfnJ4vFotWrVxd43m3btsnZ2Vlt2rQxrT4AAAAAAG6WsyNXnp6erqCgIA0aNEh9+/Yt8Hznz59XVFSU7rrrLiUnJ5tYIQAAKCssFkdXABScYTi6AgBFyaHBu2fPnurZs+cNz/foo48qMjJSFSpUuKG95AAAAAAAFLdSd473ggULdPToUU2cOLFA/TMyMpSWlmY3AQAAAABQXEpV8D506JDGjh2r//f//p+cnQu2sz4uLk6enp62KSAgwOQqAQAAAAD4n1ITvLOyshQZGanJkyercePGBZ5v3LhxSk1NtU0nTpwwsUoAAAAAAOw59BzvG3HhwgXt3r1b33//vUaMGCFJys7OlmEYcnZ21ldffaU777wzx3xWq1VWq7W4ywUAAAAAQFIpCt4eHh768ccf7dreeecdff3111q5cqXq1avnoMoAAAAAAMibQ4P3xYsXdfjwYdvjY8eOKSEhQdWrV1edOnU0btw4nTx5Uh988IGcnJzUsmVLu/m9vb3l6uqaox0AAAAAgJLCocF79+7d6tq1q+1xbGysJCk6OloLFy7U6dOnlZiY6KjyAAAAAAC4aRbDMAxHF1Gc0tLS5OnpqdTUVHl4eDi6HAD5WWJxdAVAwUUW3XDKWHVVUb8OFj5SUIqUr2/oQOl0I+NUqbmqOQAAAAAApRHBGwAAAAAAExG8AQAAAAAwEcEbAAAAAAATEbwBAAAAADARwRsAAAAAABMRvAEAAAAAMBHBGwAAFMisWbMUGBgoV1dXhYaGateuXfn2nzlzppo0aSI3NzcFBARo9OjR+uuvv4qpWgAASg6CNwAAuK7ly5crNjZWEydO1N69exUUFKTw8HCdOXMm1/5LlizR2LFjNXHiRO3fv1/vv/++li9frueee66YKwcAwPEI3gAA4LpmzJihoUOHKiYmRs2bN9ecOXNUqVIlzZ8/P9f+27dvV4cOHRQZGanAwEB1795dAwcOvO5ecgAAyiKCNwAAyFdmZqb27NmjsLAwW5uTk5PCwsK0Y8eOXOe5/fbbtWfPHlvQPnr0qNasWaNevXrluZ6MjAylpaXZTQAAlAXOji4AAACUbCkpKcrKypKPj49du4+Pj3799ddc54mMjFRKSoruuOMOGYahK1eu6NFHH833UPO4uDhNnjy5SGsHAKAkYI83AAAocps2bdKUKVP0zjvvaO/evfrkk0/0xRdf6KWXXspznnHjxik1NdU2nThxohgrBgDAPOzxBgAA+fLy8lKFChWUnJxs156cnCxfX99c53nhhRf00EMPaciQIZKkVq1aKT09XcOGDdP48ePl5JTzt3+r1Sqr1Vr0GwAAgIOxxxsAAOTLxcVFwcHBio+Pt7VlZ2crPj5e7du3z3WeS5cu5QjXFSpUkCQZhmFesQAAlEDs8QYAANcVGxur6OhotWvXTiEhIZo5c6bS09MVExMjSYqKipK/v7/i4uIkSREREZoxY4batm2r0NBQHT58WC+88IIiIiJsARwAgPKC4A0AAK5rwIABOnv2rCZMmKCkpCS1adNGa9eutV1wLTEx0W4P9/PPPy+LxaLnn39eJ0+eVM2aNRUREaFXXnnFUZsAAIDDWIxydrxXWlqaPD09lZqaKg8PD0eXAyA/SyyOrgAouMiiG04Zq64q6tfBwkcKSpHy9Q0dKJ1uZJziHG8AAAAAAExE8AYAAAAAwEQEbwAAAAAATETwBgAAAADARARvAAAAAABMRPAGAAAAAMBEBG8AAAAAAExE8AYAAAAAwEQEbwAAAAAATETwBgAAAADARARvAAAAAABMRPAGAAAAAMBEBG8AAAAAAExE8AYAAAAAwEQEbwAAAAAATETwBgAAAADARARvAAAAAABMRPAGAAAAAMBEBG8AAAAAAExE8AYAAAAAwEQEbwAAAAAATETwBgAAAADARA4N3ps3b1ZERIT8/PxksVi0evXqfPt/8skn6tatm2rWrCkPDw+1b99e69atK55iAQAAAAAoBIcG7/T0dAUFBWnWrFkF6r9582Z169ZNa9as0Z49e9S1a1dFRETo+++/N7lSAAAAAAAKx9mRK+/Zs6d69uxZ4P4zZ860ezxlyhR9+umn+vzzz9W2bdsirg4AAAAAgJvn0OB9s7Kzs3XhwgVVr149zz4ZGRnKyMiwPU5LSyuO0gAAAAAAkFTKL642bdo0Xbx4Uf3798+zT1xcnDw9PW1TQEBAMVYIAAAAACjvSm3wXrJkiSZPnqyPPvpI3t7eefYbN26cUlNTbdOJEyeKsUoAAAAAQHlXKg81X7ZsmYYMGaIVK1YoLCws375Wq1VWq7WYKgMAAAAAwF6p2+O9dOlSxcTEaOnSpbr77rsdXQ4AAAAAAPly6B7vixcv6vDhw7bHx44dU0JCgqpXr646depo3LhxOnnypD744ANJVw8vj46O1ptvvqnQ0FAlJSVJktzc3OTp6emQbQAAAAAAID8O3eO9e/dutW3b1nYrsNjYWLVt21YTJkyQJJ0+fVqJiYm2/nPnztWVK1c0fPhw1apVyzaNGjXKIfUDAAAAAHA9Dt3j3aVLFxmGkefzCxcutHu8adMmcwsCAAAAAKCIlbpzvAEAAAAAKE0I3gAAAAAAmIjgDQAAAACAiQjeAAAAAACYiOANAAAAAICJCN4AAAAAAJiI4A0AAAAAgIkI3gAAAAAAmIjgDQAAAACAiQjeAAAAAACYiOANAAAAAICJCN4AAAAAAJiI4A0AAAAAgIkI3gAAAAAAmIjgDQAAAACAiQjeAAAAAACYiOANAAAAAICJCN4AAAAAAJiI4A0AAAAAgIkI3gAAAAAAmIjgDQAAAACAiQjeAAAAAACYiOANAAAAAICJCN4AAAAAAJiI4A0AAAAAgIkI3gAAAAAAmIjgDQAAAACAiQjeAAAAAACYiOANAAAAAICJCN4AAAAAAJiI4A0AAAAAgIkI3gAAAAAAmIjgDQAAAACAiQjeAAAAAACYiOANAAAAAICJCN4AAAAAAJiI4A0AAAAAgIkI3gAAAAAAmIjgDQAAAACAiQjeAAAAAACYyKHBe/PmzYqIiJCfn58sFotWr1593Xk2bdqkW265RVarVQ0bNtTChQtNrxMAAEizZs1SYGCgXF1dFRoaql27duXb//z58xo+fLhq1aolq9Wqxo0ba82aNcVULQAAJYdDg3d6erqCgoI0a9asAvU/duyY7r77bnXt2lUJCQl68sknNWTIEK1bt87kSgEAKN+WL1+u2NhYTZw4UXv37lVQUJDCw8N15syZXPtnZmaqW7duOn78uFauXKkDBw5o3rx58vf3L+bKAQBwPGdHrrxnz57q2bNngfvPmTNH9erV0/Tp0yVJzZo109atW/XGG28oPDzcrDIBACj3ZsyYoaFDhyomJkbS1TH5iy++0Pz58zV27Ngc/efPn69z585p+/btqlixoiQpMDCwOEsGAKDEKFXneO/YsUNhYWF2beHh4dqxY0ee82RkZCgtLc1uAgAABZeZmak9e/bYjcFOTk4KCwvLcwz+7LPP1L59ew0fPlw+Pj5q2bKlpkyZoqysrOIqGwCAEqNUBe+kpCT5+PjYtfn4+CgtLU1//vlnrvPExcXJ09PTNgUEBBRHqQAAlBkpKSnKysrKdQxOSkrKdZ6jR49q5cqVysrK0po1a/TCCy9o+vTpevnll/NcDz+WAwDKqlIVvAtj3LhxSk1NtU0nTpxwdEkAAJR52dnZ8vb21ty5cxUcHKwBAwZo/PjxmjNnTp7z8GM5AKCsKlXB29fXV8nJyXZtycnJ8vDwkJubW67zWK1WeXh42E0AAKDgvLy8VKFChVzHYF9f31znqVWrlho3bqwKFSrY2po1a6akpCRlZmbmOg8/lgMAyqpSFbzbt2+v+Ph4u7b169erffv2DqoIAICyz8XFRcHBwXZjcHZ2tuLj4/Mcgzt06KDDhw8rOzvb1nbw4EHVqlVLLi4uuc7Dj+UAgLLKocH74sWLSkhIUEJCgqSrtwtLSEhQYmKipKu/fEdFRdn6P/roozp69KieffZZ/frrr3rnnXf00UcfafTo0Y4oHwCAciM2Nlbz5s3TokWLtH//fj322GNKT0+3XeU8KipK48aNs/V/7LHHdO7cOY0aNUoHDx7UF198oSlTpmj48OGO2gQAABzGobcT2717t7p27Wp7HBsbK0mKjo7WwoULdfr0aVsIl6R69erpiy++0OjRo/Xmm2+qdu3aeu+997iVGAAAJhswYIDOnj2rCRMmKCkpSW3atNHatWttF1xLTEyUk9P/fs8PCAjQunXrNHr0aLVu3Vr+/v4aNWqUxowZ46hNAADAYSyGYRiOLqI4paWlydPTU6mpqRzCBpR0SyyOrgAouMiiG04Zq64q6tfBwkcKSpHy9Q0dKJ1uZJwqVed4AwAAAABQ2hC8AQAAAAAwEcEbAAAAAAATEbwBAAAAADARwRsAAAAAABMRvAEAAAAAMBHBGwAAAAAAExG8AQAoowIDA/Xiiy8qMTHR0aUAAFCuEbwBACijnnzySX3yySeqX7++unXrpmXLlikjI8PRZQEAUO4QvAEAKKOefPJJJSQkaNeuXWrWrJmeeOIJ1apVSyNGjNDevXsdXR4AAOUGwRsAgDLulltu0VtvvaVTp05p4sSJeu+993TrrbeqTZs2mj9/vgzDcHSJAACUac6OLgAAAJjr8uXLWrVqlRYsWKD169frtttu0+DBg/X777/rueee04YNG7RkyRJHlwkAQJlF8AYAoIzau3evFixYoKVLl8rJyUlRUVF644031LRpU1ufe++9V7feeqsDqwQAoOwjeAMAUEbdeuut6tatm2bPnq0+ffqoYsWKOfrUq1dPDzzwgAOqAwCg/CB4AwBQRh09elR169bNt0/lypW1YMGCYqoIAIDyiYurAQBQRp05c0Y7d+7M0b5z507t3r3bARUBAFA+EbwBACijhg8frhMnTuRoP3nypIYPH+6AigAAKJ8I3gAAlFG//PKLbrnllhztbdu21S+//OKAigAAKJ8I3gAAlFFWq1XJyck52k+fPi1nZy7zAgBAcSF4AwBQRnXv3l3jxo1Tamqqre38+fN67rnn1K1bNwdWBgBA+cLP3QAAlFHTpk1Tp06dVLduXbVt21aSlJCQIB8fH3344YcOrg4AgPKD4A0AQBnl7++vffv2afHixfrhhx/k5uammJgYDRw4MNd7egMAAHMQvAEAKMMqV66sYcOGOboMAADKNYI3AABl3C+//KLExERlZmbatd9zzz0OqggAgPKlUMH7xIkTslgsql27tiRp165dWrJkiZo3b86v6gAAlBBHjx7Vvffeqx9//FEWi0WGYUiSLBaLJCkrK8uR5QEAUG4U6qrmkZGR2rhxoyQpKSlJ3bp1065duzR+/Hi9+OKLRVogAAAonFGjRqlevXo6c+aMKlWqpJ9//lmbN29Wu3bttGnTJkeXBwBAuVGo4P3TTz8pJCREkvTRRx+pZcuW2r59uxYvXqyFCxcWZX0AAKCQduzYoRdffFFeXl5ycnKSk5OT7rjjDsXFxWnkyJGOLg8AgHKjUMH78uXLslqtkqQNGzbYzhFr2rSpTp8+XXTVAQCAQsvKylKVKlUkSV5eXjp16pQkqW7dujpw4IAjSwMAoFwpVPBu0aKF5syZoy1btmj9+vXq0aOHJOnUqVOqUaNGkRYIAAAKp2XLlvrhhx8kSaGhoZo6daq2bdumF198UfXr13dwdQAAlB+FCt6vvfaa3n33XXXp0kUDBw5UUFCQJOmzzz6zHYIOAAAc6/nnn1d2drYk6cUXX9SxY8fUsWNHrVmzRm+99ZaDqwMAoPwo1FXNu3TpopSUFKWlpalatWq29mHDhqlSpUpFVhwAACi88PBw278bNmyoX3/9VefOnVO1atVsVzYHAADmK9Qe7z///FMZGRm20P3bb79p5syZOnDggLy9vYu0QAAAcOMuX74sZ2dn/fTTT3bt1atXJ3QDAFDMChW8e/furQ8++ECSdP78eYWGhmr69Onq06ePZs+eXaQFAgCAG1exYkXVqVOHe3UDAFACFCp47927Vx07dpQkrVy5Uj4+Pvrtt9/0wQcfcM4YAAAlxPjx4/Xcc8/p3Llzji4FAIByrVDneF+6dMl2e5KvvvpKffv2lZOTk2677Tb99ttvRVogAAAonLfffluHDx+Wn5+f6tatq8qVK9s9v3fvXgdVBgBA+VKo4N2wYUOtXr1a9957r9atW6fRo0dLks6cOSMPD48iLRAAABROnz59HF0CAABQIYP3hAkTFBkZqdGjR+vOO+9U+/btJV3d+922bdsiLRAAABTOxIkTHV0CAABQIYN3v379dMcdd+j06dO2e3hL0l133aV77723yIoDAAAAAKC0K1TwliRfX1/5+vrq999/lyTVrl1bISEhRVYYAAC4OU5OTvneOowrngMAUDwKFbyzs7P18ssva/r06bp48aIkqUqVKnrqqac0fvx4OTkV6mLpAACgCK1atcru8eXLl/X9999r0aJFmjx5soOqAgCg/ClU8B4/frzef/99vfrqq+rQoYMkaevWrZo0aZL++usvvfLKKze0vFmzZun1119XUlKSgoKC9O9//zvfveczZ87U7NmzlZiYKC8vL/Xr109xcXFydXUtzOYAAFAm9e7dO0dbv3791KJFCy1fvlyDBw92QFUAAJQ/hQreixYt0nvvvad77rnH1ta6dWv5+/vr8ccfv6HgvXz5csXGxmrOnDkKDQ3VzJkzFR4ergMHDsjb2ztH/yVLlmjs2LGaP3++br/9dh08eFAPP/ywLBaLZsyYUZjNAQCgXLnttts0bNgwR5cBAEC5Uahjws+dO6emTZvmaG/atKnOnTt3Q8uaMWOGhg4dqpiYGDVv3lxz5sxRpUqVNH/+/Fz7b9++XR06dFBkZKQCAwPVvXt3DRw4ULt27SrMpgAAUK78+eefeuutt+Tv7+/oUgAAKDcKFbyDgoL09ttv52h/++231bp16wIvJzMzU3v27FFYWNj/CnJyUlhYmHbs2JHrPLfffrv27NljC9pHjx7VmjVr1KtXr1z7Z2RkKC0tzW4CAKA8qFatmqpXr26bqlWrpipVqmj+/Pl6/fXXHV0eAADlRqEONZ86daruvvtubdiwwXYP7x07dujEiRNas2ZNgZeTkpKirKws+fj42LX7+Pjo119/zXWeyMhIpaSk6I477pBhGLpy5YoeffRRPffcc7n2j4uL4wIyAIBy6Y033rC7qrmTk5Nq1qyp0NBQVatWzYGVAQBQvhQqeHfu3FkHDx7UrFmzbAG5b9++GjZsmF5++WV17NixSIv8u02bNmnKlCl65513FBoaqsOHD2vUqFF66aWX9MILL+ToP27cOMXGxtoep6WlKSAgwLT6AAAoKR5++GFHlwAAAHQT9/H28/PLcRG1H374Qe+//77mzp1boGV4eXmpQoUKSk5OtmtPTk6Wr69vrvO88MILeuihhzRkyBBJUqtWrZSenq5hw4bleiszq9Uqq9Va0M0CAKDMWLBggdzd3XX//ffbta9YsUKXLl1SdHS0gyoDAKB8cegNt11cXBQcHKz4+HhbW3Z2tuLj422HsP/TpUuXcoTrChUqSJIMwzCvWAAASpm4uDh5eXnlaPf29taUKVMcUBEAAOVTofd4F5XY2FhFR0erXbt2CgkJ0cyZM5Wenq6YmBhJUlRUlPz9/RUXFydJioiI0IwZM9S2bVvboeYvvPCCIiIibAEcAABIiYmJqlevXo72unXrKjEx0QEVAQBQPjk8eA8YMEBnz57VhAkTlJSUpDZt2mjt2rW2C64lJiba7eF+/vnnZbFY9Pzzz+vkyZOqWbOmIiIibuje4QAAlAfe3t7at2+fAgMD7dp/+OEH1ahRwzFFAQBQDt1Q8O7bt2++z58/f75QRYwYMUIjRozI9blNmzbZPXZ2dtbEiRM1ceLEQq0LAIDyYuDAgRo5cqSqVKmiTp06SZK++eYbjRo1Sg888ICDqwMAoPy4oeDt6el53eejoqJuqiAAAFA0XnrpJR0/flx33XWXnJ2vDvnZ2dmKioriHG8AAIrRDQXvBQsWmFUHAAAoYi4uLlq+fLlefvllJSQkyM3NTa1atVLdunUdXRoAAOWKw8/xBgAA5mrUqJEaNWrk6DIAACi3HHo7MQAAYJ777rtPr732Wo72qVOn5ri3NwAAMA/BGwCAMmrz5s3q1atXjvaePXtq8+bNDqgIAIDyieANAEAZdfHiRbm4uORor1ixotLS0hxQEQAA5RPBGwCAMqpVq1Zavnx5jvZly5apefPmDqgIAIDyiYurAQBQRr3wwgvq27evjhw5ojvvvFOSFB8fryVLlmjlypUOrg4AgPKD4A0AQBkVERGh1atXa8qUKVq5cqXc3NwUFBSkr7/+WtWrV3d0eQAAlBsEbwAAyrC7775bd999tyQpLS1NS5cu1dNPP609e/YoKyvLwdUBAFA+cI43AABl3ObNmxUdHS0/Pz9Nnz5dd955p7799ltHlwUAQLnBHm8AAMqgpKQkLVy4UO+//77S0tLUv39/ZWRkaPXq1VxYDQCAYsYebwAAypiIiAg1adJE+/bt08yZM3Xq1Cn9+9//dnRZAACUW+zxBgCgjPnyyy81cuRIPfbYY2rUqJGjywEAoNxjjzcAAGXM1q1bdeHCBQUHBys0NFRvv/22UlJSHF0WAADlFsEbAIAy5rbbbtO8efN0+vRpPfLII1q2bJn8/PyUnZ2t9evX68KFC44uEQCAcoXgDQBAGVW5cmUNGjRIW7du1Y8//qinnnpKr776qry9vXXPPfc4ujwAAMoNgjcAAOVAkyZNNHXqVP3+++9aunSpo8sBAKBcIXgDAFCOVKhQQX369NFnn33m6FIAACg3CN4AAAAAAJiI4A0AAAAAgIkI3gAAAAAAmIjgDQAAAACAiQjeAACgQGbNmqXAwEC5uroqNDRUu3btKtB8y5Ytk8ViUZ8+fcwtEACAEorgDQAArmv58uWKjY3VxIkTtXfvXgUFBSk8PFxnzpzJd77jx4/r6aefVseOHYupUgAASh6CNwAAuK4ZM2Zo6NChiomJUfPmzTVnzhxVqlRJ8+fPz3OerKwsPfjgg5o8ebLq169fjNUCAFCyELwBAEC+MjMztWfPHoWFhdnanJycFBYWph07duQ534svvihvb28NHjy4OMoEAKDEcnZ0AQAAoGRLSUlRVlaWfHx87Np9fHz066+/5jrP1q1b9f777yshIaHA68nIyFBGRobtcVpaWqHqBQCgpGGPNwAAKFIXLlzQQw89pHnz5snLy6vA88XFxcnT09M2BQQEmFglAADFhz3eAAAgX15eXqpQoYKSk5Pt2pOTk+Xr65uj/5EjR3T8+HFFRETY2rKzsyVJzs7OOnDggBo0aJBjvnHjxik2Ntb2OC0tjfANACgTCN4AACBfLi4uCg4OVnx8vO2WYNnZ2YqPj9eIESNy9G/atKl+/PFHu7bnn39eFy5c0JtvvplnmLZarbJarUVePwAAjkbwBgAA1xUbG6vo6Gi1a9dOISEhmjlzptLT0xUTEyNJioqKkr+/v+Li4uTq6qqWLVvazV+1alVJytEOAEB5QPAGAADXNWDAAJ09e1YTJkxQUlKS2rRpo7Vr19ouuJaYmCgnJy4dAwBAbiyGYRiOLqI4paWlydPTU6mpqfLw8HB0OQDys8Ti6AqAgossuuGUseqqon4dLHykoBQpX9/QgdLpRsYpfpoGAAAAAMBEBG8AAAAAAExE8AYAAAAAwEQEbwAAAAAATETwBgAAAADARARvAAAAAABMVCKC96xZsxQYGChXV1eFhoZq165d+fY/f/68hg8frlq1aslqtapx48Zas2ZNMVULAAAAAEDBOTu6gOXLlys2NlZz5sxRaGioZs6cqfDwcB04cEDe3t45+mdmZqpbt27y9vbWypUr5e/vr99++01Vq1Yt/uIBAAAAALgOhwfvGTNmaOjQoYqJiZEkzZkzR1988YXmz5+vsWPH5ug/f/58nTt3Ttu3b1fFihUlSYGBgcVZMgAAAAAABebQQ80zMzO1Z88ehYWF2dqcnJwUFhamHTt25DrPZ599pvbt22v48OHy8fFRy5YtNWXKFGVlZeXaPyMjQ2lpaXYTAAAAAADFxaHBOyUlRVlZWfLx8bFr9/HxUVJSUq7zHD16VCtXrlRWVpbWrFmjF154QdOnT9fLL7+ca/+4uDh5enrapoCAgCLfDgAAAAAA8lIiLq52I7Kzs+Xt7a25c+cqODhYAwYM0Pjx4zVnzpxc+48bN06pqam26cSJE8VcMQAAAACgPHPoOd5eXl6qUKGCkpOT7dqTk5Pl6+ub6zy1atVSxYoVVaFCBVtbs2bNlJSUpMzMTLm4uNj1t1qtslqtRV88AAAAAAAF4NA93i4uLgoODlZ8fLytLTs7W/Hx8Wrfvn2u83To0EGHDx9Wdna2re3gwYOqVatWjtANAAAAAICjOfxQ89jYWM2bN0+LFi3S/v379dhjjyk9Pd12lfOoqCiNGzfO1v+xxx7TuXPnNGrUKB08eFBffPGFpkyZouHDhztqEwAAAAAAyJPDbyc2YMAAnT17VhMmTFBSUpLatGmjtWvX2i64lpiYKCen//0+EBAQoHXr1mn06NFq3bq1/P39NWrUKI0ZM8ZRmwAAAAAAQJ4shmEYji6iOKWlpcnT01Opqany8PBwdDkA8rPE4ugKgIKLLLrhlLHqqqJ+HSx8pKAUKV/f0IHS6UbGKYcfag4AAAAAQFlG8AYAAAAAwEQEbwAAAAAATETwBgAAAADARARvAAAAAABMRPAGAAAAAMBEBG8AAAAAAExE8AYAAAAAwEQEbwAAAAAATETwBgAAAADARARvAAAAAABMRPAGAAAAAMBEBG8AAAAAAExE8AYAAAAAwEQEbwAAAAAATETwBgAAAADARARvAAAAAABM5OzoAgAAAACUYkssjq4AuDGRRrGvkj3eAAAAAACYiOANAAAAAICJCN4AAAAAAJiI4A0AAAAAgIkI3gAAAAAAmIjgDQAAAACAiQjeAAAAAACYiOANAAAAAICJCN4AAAAAAJiI4A0AAAAAgIkI3gAAAAAAmIjgDQAAAACAiQjeAAAAAACYiOANAAAAAICJCN4AAAAAAJiI4A0AAAAAgIkI3gAAAAAAmIjgDQAAAACAiQjeAAAAAACYiOANAAAAAICJCN4AAAAAAJiI4A0AAAAAgIlKRPCeNWuWAgMD5erqqtDQUO3atatA8y1btkwWi0V9+vQxt8DrsFiYmErPBAAAAKB4OTx4L1++XLGxsZo4caL27t2roKAghYeH68yZM/nOd/z4cT399NPq2LFjMVUKAAAAAMCNc3jwnjFjhoYOHaqYmBg1b95cc+bMUaVKlTR//vw858nKytKDDz6oyZMnq379+sVYLQAAAAAAN8ahwTszM1N79uxRWFiYrc3JyUlhYWHasWNHnvO9+OKL8vb21uDBg6+7joyMDKWlpdlNAAAAAAAUF4cG75SUFGVlZcnHx8eu3cfHR0lJSbnOs3XrVr3//vuaN29egdYRFxcnT09P2xQQEHDTdQMAAAAAUFAOP9T8Rly4cEEPPfSQ5s2bJy8vrwLNM27cOKWmptqmEydOmFwlAAAAAAD/4+zIlXt5ealChQpKTk62a09OTpavr2+O/keOHNHx48cVERFha8vOzpYkOTs768CBA2rQoIHdPFarVVar1YTqAQAAAAC4Pofu8XZxcVFwcLDi4+NtbdnZ2YqPj1f79u1z9G/atKl+/PFHJSQk2KZ77rlHXbt2VUJCAoeRAwAAAABKHIfu8Zak2NhYRUdHq127dgoJCdHMmTOVnp6umJgYSVJUVJT8/f0VFxcnV1dXtWzZ0m7+qlWrSlKOdgAAAAAASgKHB+8BAwbo7NmzmjBhgpKSktSmTRutXbvWdsG1xMREOTmVqlPRAQAAAACwsRiGYTi6iOKUlpYmT09PpaamysPDo0iWabEUyWKAYlGq3vFLeHOhFIksujeXGWNVaVTUrwPjNUoTxmvAREU0Zt/IOMWuZAAAAAAATETwBgAAAADARARvAAAAAABMRPAGAAAAAMBEBG8AAAAAAExE8AYAAAAAwEQEbwAAUCCzZs1SYGCgXF1dFRoaql27duXZd968eerYsaOqVaumatWqKSwsLN/+AACUZQRvAABwXcuXL1dsbKwmTpyovXv3KigoSOHh4Tpz5kyu/Tdt2qSBAwdq48aN2rFjhwICAtS9e3edPHmymCsHAMDxCN4AAOC6ZsyYoaFDhyomJkbNmzfXnDlzVKlSJc2fPz/X/osXL9bjjz+uNm3aqGnTpnrvvfeUnZ2t+Pj4Yq4cAADHI3gDAIB8ZWZmas+ePQoLC7O1OTk5KSwsTDt27CjQMi5duqTLly+revXqefbJyMhQWlqa3QQAQFlA8AYAAPlKSUlRVlaWfHx87Np9fHyUlJRUoGWMGTNGfn5+duH9n+Li4uTp6WmbAgICbqpuAABKCoI3AAAw1auvvqply5Zp1apVcnV1zbPfuHHjlJqaaptOnDhRjFUCAGAeZ0cXAAAASjYvLy9VqFBBycnJdu3Jycny9fXNd95p06bp1Vdf1YYNG9S6det8+1qtVlmt1puuFwCAkoY93gAAIF8uLi4KDg62uzDatQultW/fPs/5pk6dqpdeeklr165Vu3btiqNUAABKJPZ4AwCA64qNjVV0dLTatWunkJAQzZw5U+np6YqJiZEkRUVFyd/fX3FxcZKk1157TRMmTNCSJUsUGBhoOxfc3d1d7u7uDtsOAAAcgeANAACua8CAATp79qwmTJigpKQktWnTRmvXrrVdcC0xMVFOTv87kG727NnKzMxUv3797JYzceJETZo0qThLBwDA4QjeAACgQEaMGKERI0bk+tymTZvsHh8/ftz8ggAAKCU4xxsAAAAAABMRvAEAAAAAMBHBGwAAAAAAExG8AQAAAAAwEcEbAAAAAAATEbwBAAAAADARwRsAAAAAABMRvAEAAAAAMBHBGwAAAAAAExG8AQAAAAAwEcEbAAAAAAATEbwBAAAAADARwRsAAAAAABMRvAEAAAAAMBHBGwAAAAAAExG8AQAAAAAwEcEbAAAAAAATEbwBAAAAADARwRsAAAAAABMRvAEAAAAAMBHBGwAAAAAAExG8AQAAAAAwEcEbAAAAAAATlYjgPWvWLAUGBsrV1VWhoaHatWtXnn3nzZunjh07qlq1aqpWrZrCwsLy7Q8AAAAAgCM5PHgvX75csbGxmjhxovbu3augoCCFh4frzJkzufbftGmTBg4cqI0bN2rHjh0KCAhQ9+7ddfLkyWKuHAAAAACA63N48J4xY4aGDh2qmJgYNW/eXHPmzFGlSpU0f/78XPsvXrxYjz/+uNq0aaOmTZvqvffeU3Z2tuLj44u5cgAAAAAArs+hwTszM1N79uxRWFiYrc3JyUlhYWHasWNHgZZx6dIlXb58WdWrV8/1+YyMDKWlpdlNAAAAAAAUF4cG75SUFGVlZcnHx8eu3cfHR0lJSQVaxpgxY+Tn52cX3v8uLi5Onp6etikgIOCm6wYAAAAAoKAcfqj5zXj11Ve1bNkyrVq1Sq6urrn2GTdunFJTU23TiRMnirlKAAAAAEB55uzIlXt5ealChQpKTk62a09OTpavr2++806bNk2vvvqqNmzYoNatW+fZz2q1ymq1Fkm9AAAAAADcKIfu8XZxcVFwcLDdhdGuXSitffv2ec43depUvfTSS1q7dq3atWtXHKUCAAAAAFAoDt3jLUmxsbGKjo5Wu3btFBISopkzZyo9PV0xMTGSpKioKPn7+ysuLk6S9Nprr2nChAlasmSJAgMDbeeCu7u7y93d3WHbAQAAAABAbhwevAcMGKCzZ89qwoQJSkpKUps2bbR27VrbBdcSExPl5PS/HfOzZ89WZmam+vXrZ7eciRMnatKkScVZOgAAAAAA1+Xw4C1JI0aM0IgRI3J9btOmTXaPjx8/bn5BAAAAAAAUkVJ9VXMAAAAAAEo6gjcAAAAAACYieAMAAAAAYCKCNwAAAAAAJiJ4AwAAAABgIoI3AAAAAAAmIngDAAAAAGAigjcAAAAAACYieAMAAAAAYCKCNwAAAAAAJiJ4AwAAAABgIoI3AAAAAAAmIngDAAAAAGAigjcAAAAAACYieAMAAAAAYCKCNwAAAAAAJiJ4AwAAAABgIoI3AAAAAAAmIngDAAAAAGAigjcAAAAAACYieAMAAAAAYCKCNwAAAAAAJiJ4AwAAAABgIoI3AAAAAAAmIngDAAAAAGAigjcAAAAAACYieAMAAAAAYCKCNwAAAAAAJiJ4AwAAAABgIoI3AAAAAAAmIngDAAAAAGAigjcAAAAAACYieAMAAAAAYCKCNwAAAAAAJiJ4AwAAAABgIoI3AAAAAAAmIngDAAAAAGAigjcAAAAAACYieAMAAAAAYCKCNwAAAAAAJioRwXvWrFkKDAyUq6urQkNDtWvXrnz7r1ixQk2bNpWrq6tatWqlNWvWFFOlAACUX4zXAAAUjsOD9/LlyxUbG6uJEydq7969CgoKUnh4uM6cOZNr/+3bt2vgwIEaPHiwvv/+e/Xp00d9+vTRTz/9VMyVAwBQfjBeAwBQeBbDMAxHFhAaGqpbb71Vb7/9tiQpOztbAQEBeuKJJzR27Ngc/QcMGKD09HT95z//sbXddtttatOmjebMmXPd9aWlpcnT01Opqany8PAokm2wWIpkMUCxcOw7/gYt4c2FUiSy6N5cZoxVN6u4x2up6F8HxmuUJozXgImKaMy+kXHKuUjWWEiZmZnas2ePxo0bZ2tzcnJSWFiYduzYkes8O3bsUGxsrF1beHi4Vq9enWv/jIwMZWRk2B6npqZKuvoiAeVRqfrTv+ToAoAbUIRvrmtjlIN/G7cpjvFaYswG/q5U/dkzXqO0KaI32I2M1w4N3ikpKcrKypKPj49du4+Pj3799ddc50lKSsq1f1JSUq794+LiNHny5BztAQEBhawaKN08PR1dAVBGDS36N9eFCxfkWQLetMUxXkuM2cDflYC3PlB2FfGYXZDx2qHBuziMGzfO7hf37OxsnTt3TjVq1JCFY85KrLS0NAUEBOjEiRMl5jBLoCzgvVU6GIahCxcuyM/Pz9GlFCvG7NKHzxTAPLy/Sr4bGa8dGry9vLxUoUIFJScn27UnJyfL19c313l8fX1vqL/VapXVarVrq1q1auGLRrHy8PDggwYwAe+tkq8k7Om+pjjGa4kxuzTjMwUwD++vkq2g47VDr2ru4uKi4OBgxcfH29qys7MVHx+v9u3b5zpP+/bt7fpL0vr16/PsDwAAbg7jNQAAN8fhh5rHxsYqOjpa7dq1U0hIiGbOnKn09HTFxMRIkqKiouTv76+4uDhJ0qhRo9S5c2dNnz5dd999t5YtW6bdu3dr7ty5jtwMAADKNMZrAAAKz+HBe8CAATp79qwmTJigpKQktWnTRmvXrrVdkCUxMVFOTv/bMX/77bdryZIlev755/Xcc8+pUaNGWr16tVq2bOmoTYAJrFarJk6cmOOQQwA3h/cWCovxGrnhMwUwD++vssXh9/EGAAAAAKAsc+g53gAAAAAAlHUEbwAAAAAATETwBgAAAADARARv3JDAwEDNnDnT9PV06dJFTz75pOnruR6LxaLVq1cXuP/DDz+sPn36mFYPcKN/k4VVXO/1/Bw/flwWi0UJCQkFnqekfHYAJQFjdv4Ys2Emxuv8lZTPjeJE8C6jLBZLvtOkSZMKtdzvvvtOw4YNK9piC2HhwoWyWCxq1qxZjudWrFghi8WiwMDA4i8MZUJeg8HChQtVtWrVG1pWUQ+Ip0+fVs+ePYtseYU1adIkWSwW9ejRI8dzr7/+uiwWi7p06VL8hQGlEGM2YzYKh/H6+hivSw6H304M5jh9+rTt38uXL9eECRN04MABW5u7u7vt34ZhKCsrS87O1/9zqFmzZtEWehMqV66sM2fOaMeOHWrfvr2t/f3331edOnUcWBlwY7KysmSxWOxuxZQXX1/fYqioYGrVqqWNGzfq999/V+3atW3t8+fP5z0I3ADGbD4vUDowXuNmsMe7jPL19bVNnp6eslgstse//vqrqlSpoi+//FLBwcGyWq3aunWrjhw5ot69e8vHx0fu7u669dZbtWHDBrvl/vPXQIvFovfee0/33nuvKlWqpEaNGumzzz6zm+enn35Sz5495e7uLh8fHz300ENKSUmxPZ+enq6oqCi5u7urVq1amj59eoG20dnZWZGRkZo/f76t7ffff9emTZsUGRmZo//s2bPVoEEDubi4qEmTJvrwww/tnj906JA6deokV1dXNW/eXOvXr8+xjBMnTqh///6qWrWqqlevrt69e+v48eMFqhdlz7XDFKdNm6ZatWqpRo0aGj58uC5fvizp6i/xv/32m0aPHm3bcyX975f4zz77TM2bN5fValViYqK+++47devWTV5eXvL09FTnzp21d+9eu3X+/dC1a4d2ffLJJ+ratasqVaqkoKAg7dixw26erVu3qmPHjnJzc1NAQIBGjhyp9PR02/NnzpxRRESE3NzcVK9ePS1evLhA2+/t7a3u3btr0aJFtrbt27crJSVFd999t13f7Oxsvfjii6pdu7asVqvtHtB/t2vXLrVt21aurq5q166dvv/++xzrvN7nCVAaMWYzZsNcjNeM1yUBwbscGzt2rF599VXt379frVu31sWLF9WrVy/Fx8fr+++/V48ePRQREaHExMR8lzN58mT1799f+/btU69evfTggw/q3LlzkqTz58/rzjvvVNu2bbV7926tXbtWycnJ6t+/v23+Z555Rt98840+/fRTffXVV9q0aVOOD6+8DBo0SB999JEuXbok6eoHZI8ePeTj42PXb9WqVRo1apSeeuop/fTTT3rkkUcUExOjjRs3Srr6IdO3b1+5uLho586dmjNnjsaMGWO3jMuXLys8PFxVqlTRli1btG3bNrm7u6tHjx7KzMwsUL0oezZu3KgjR45o48aNWrRokRYuXKiFCxdKkj755BPVrl1bL774ok6fPm23V+vSpUt67bXX9N577+nnn3+Wt7e3Lly4oOjoaG3dulXffvutGjVqpF69eunChQv51jB+/Hg9/fTTSkhIUOPGjTVw4EBduXJFknTkyBH16NFD9913n/bt26fly5dr69atGjFihG3+hx9+WCdOnNDGjRu1cuVKvfPOOzpz5kyBtn/QoEG27ZWu/nr+4IMPysXFxa7fm2++qenTp2vatGnat2+fwsPDdc899+jQoUOSpIsXL+pf//qXmjdvrj179mjSpEl6+umn7ZZRkM8ToKxizGbMxs1hvGa8djgDZd6CBQsMT09P2+ONGzcakozVq1dfd94WLVoY//73v22P69ata7zxxhu2x5KM559/3vb44sWLhiTjyy+/NAzDMF566SWje/fudss8ceKEIck4cOCAceHCBcPFxcX46KOPbM//8ccfhpubmzFq1KgCbVObNm2MRYsWGdnZ2UaDBg2MTz/91HjjjTeMunXr2vrffvvtxtChQ+2Wcf/99xu9evUyDMMw1q1bZzg7OxsnT560Pf/ll18akoxVq1YZhmEYH374odGkSRMjOzvb1icjI8Nwc3Mz1q1bZxiGYURHRxu9e/fOs26UDp07d8717++f76Xo6Gijbt26xpUrV2xt999/vzFgwADb43++Z64tR5KRkJCQbx1ZWVlGlSpVjM8//9zW9ve/yWPHjhmSjPfee8/2/M8//2xIMvbv328YhmEMHjzYGDZsmN1yt2zZYjg5ORl//vmnceDAAUOSsWvXLtvz+/fvNyTlqPvvJk6caAQFBRmZmZmGt7e38c033xgXL140qlSpYvzwww/GqFGjjM6dO9v6+/n5Ga+88ordMm699Vbj8ccfNwzDMN59912jRo0axp9//ml7fvbs2YYk4/vvvzcM4/qfJ4aR9/8dUFowZjNmo+AYrxmvSxP2eJdj7dq1s3t88eJFPf3002rWrJmqVq0qd3d37d+//7q/nrdu3dr278qVK8vDw8P269sPP/ygjRs3yt3d3TY1bdpU0tVf9o4cOaLMzEyFhoballG9enU1adKkwNsxaNAgLViwQN98843S09PVq1evHH3279+vDh062LV16NBB+/fvtz0fEBAgPz8/2/N/Pwft2rYcPnxYVapUsW1L9erV9ddff+nIkSMFrhdlS4sWLVShQgXb41q1ahXo12cXFxe7944kJScna+jQoWrUqJE8PT3l4eGhixcv3tB7sFatWpJk9x5cuHCh3XswPDxc2dnZOnbsmPbv3y9nZ2cFBwfbltG0adMCX5SmYsWK+r//+z8tWLBAK1asUOPGjXNsV1pamk6dOnXd92Dr1q3l6upqez6392B+nydAWcaYzZiNm8N4zXjtaFxcrRyrXLmy3eOnn35a69ev17Rp09SwYUO5ubmpX79+1z0kq2LFinaPLRaLsrOzJV39YhAREaHXXnstx3y1atXS4cOHb3IrpAcffFDPPvusJk2apIceeqhAF5wpjIsXLyo4ODjX82lK0gVscPM8PDyUmpqao/38+fPy9PS0a8vv7z8/bm5utnPIromOjtYff/yhN998U3Xr1pXValX79u1v6D14bZl/fw8+8sgjGjlyZI756tSpo4MHD1631usZNGiQQkND9dNPP2nQoEE3vby8XO/zBCjLGLNvDGN2+cB4fWMYrx2L4A2bbdu26eGHH9a9994r6eqb5mYvQnLLLbfo448/VmBgYK6Da4MGDVSxYkXt3LnTdlXF//73vzp48KA6d+5coHVUr15d99xzjz766CPNmTMn1z7NmjXTtm3bFB0dbWvbtm2bmjdvbnv+xIkTOn36tO0D4dtvv82xLcuXL5e3t7c8PDwKVBtKpyZNmuirr77K0b537141btz4hpbl4uKirKysAvXdtm2b3nnnHdseoBMnTtz0hUhuueUW/fLLL2rYsGGuzzdt2lRXrlzRnj17dOutt0qSDhw4oPPnzxd4HS1atFCLFi20b9++XC+S5OHhIT8/P23bts3ufb1t2zaFhIRIuvoe/PDDD/XXX3/ZfkXP7T2Y3+cJUJ4wZjNmg/Ga8bp04VBz2DRq1EiffPKJEhIS9MMPPygyMrJAvwTmZ/jw4Tp37pwGDhyo7777TkeOHNG6desUExOjrKwsubu7a/DgwXrmmWf09ddf66efftLDDz9coNs0/N3ChQuVkpJiO4zln5555hktXLhQs2fP1qFDhzRjxgx98skntotBhIWFqXHjxoqOjtYPP/ygLVu2aPz48XbLePDBB+Xl5aXevXtry5YtOnbsmDZt2qSRI0fq999/L9wLhBLpscce08GDBzVy5Ejt27dPBw4c0IwZM7R06VI99dRTN7SswMBAbd68WSdPnrzuoNyoUSN9+OGH2r9/v3bu3KkHH3xQbm5uN7MpGjNmjLZv364RI0YoISFBhw4d0qeffmq7WEuTJk3Uo0cPPfLII9q5c6f27NmjIUOG3PB6v/76a50+fTrPQ96eeeYZvfbaa1q+fLkOHDigsWPHKiEhQaNGjZIkRUZGymKxaOjQofrll1+0Zs0aTZs2zW4Z1/s8AcoTxmzGbDBeM16XLgRv2MyYMUPVqlXT7bffroiICIWHh+uWW265qWVe+9UsKytL3bt3V6tWrfTkk0+qatWqtoH69ddfV8eOHRUREaGwsDDdcccdduevFISbm5tq1KiR5/N9+vTRm2++qWnTpqlFixZ69913tWDBAnXp0kWS5OTkpFWrVunPP/9USEiIhgwZoldeecVuGZUqVdLmzZtVp04d9e3bV82aNdPgwYP1119/8Wt6GVO/fn1t3rxZv/76q8LCwhQaGqqPPvpIK1asUI8ePW5oWS+++KKOHz+uBg0aXPfwxvfff1///e9/dcstt+ihhx7SyJEj5e3tfTObotatW+ubb77RwYMH1bFjR7Vt21YTJkywOzdywYIF8vPzU+fOndW3b18NGzbshtdbuXLlfM8zGzlypGJjY/XUU0+pVatWWrt2rT777DM1atRI0tX7FH/++ef68ccf1bZtW40fPz7HIWoF+TwBygvGbMZsMF4zXpcuFsMwDEcXAQAAAABAWVV+f3IAAAAAAKAYELwBAAAAADARwRsAAAAAABMRvAEAAAAAMBHBGwAAAAAAExG8AQAAAAAwEcEbAAAAAAATEbwBAAAAADARwRsAAAAAABMRvAEAAAAAMBHBGwAAAAAAExG8AQAAAAAw0f8Hy5RD7YwjGp8AAAAASUVORK5CYII=\n" + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ] + }, + "metadata": {} + } + ] + }, + { + "cell_type": "code", + "source": [], + "metadata": { + "id": "v0W2cpsWblUc" + }, + "id": "v0W2cpsWblUc", + "execution_count": null, + "outputs": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.20" + }, + "colab": { + "provenance": [], + "gpuType": "A100" + }, + "accelerator": "GPU", + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "f559502a61384d8895066de4905d7bdf": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_af39de1ed03a47b7ad9513ef8d332a8c", + "IPY_MODEL_24d8020746bb464c8cd69bc35fa4cadf", + "IPY_MODEL_cfee43bbf82049efb6fbbe0490930f31" + ], + "layout": "IPY_MODEL_00f422a7380b488a80c69b84d8479ae4" + } + }, + "af39de1ed03a47b7ad9513ef8d332a8c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_498135911ebc4b05945003fc0396e2b3", + "placeholder": "​", + "style": "IPY_MODEL_14c23facacb440638f9ef71480066f2c", + "value": "Epoch 1/1: 100%" + } + }, + "24d8020746bb464c8cd69bc35fa4cadf": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_a0a9629453294ac6a6da8442ced3aa91", + "max": 1407, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_ba67a3d57664494d844ee60eb2951d54", + "value": 1407 + } + }, + "cfee43bbf82049efb6fbbe0490930f31": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_57f1b16a8e804b24b7710c4e47a32ba3", + "placeholder": "​", + "style": "IPY_MODEL_2788611a893f45819c9761f6d20108b8", + "value": " 1407/1407 [04:42<00:00,  1.03it/s]" + } + }, + "00f422a7380b488a80c69b84d8479ae4": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": "hidden", + "width": null + } + }, + "498135911ebc4b05945003fc0396e2b3": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "14c23facacb440638f9ef71480066f2c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "a0a9629453294ac6a6da8442ced3aa91": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "ba67a3d57664494d844ee60eb2951d54": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "57f1b16a8e804b24b7710c4e47a32ba3": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "2788611a893f45819c9761f6d20108b8": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + } + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/Level3/Dristro/model_comparision.png b/Level3/Dristro/model_comparision.png new file mode 100644 index 0000000000000000000000000000000000000000..df5034cecb72fe70884df363f83d378e3407e3d8 GIT binary patch literal 23590 zcmdtK2UJzrmNk6QQY;fyWdKybfB_{L5D@`0B1#UDMRG=RhSCxP5=5es6eJ2r&Y&Wq zAd*42Bn2dcWXb>B*XsA)=ziT_b^6Bme#R(^vhF$eoV{0=Yp%KWT`39Cb*s0mrcfyB z#4enbp-`6hQYds2t5)J~lJ?Pa;a>t)=dM`En(12EYFg+}E^Atu8=F}f8(iCNqhn!d zU}k!l`xrMb=XQN7D|5@^JUk|U`vA9@g&q%IN=5>1^0WB`B})p0L6iJL7b_BDK%uBS z6gzuL&MsuA)&7c{_0rO4W5l7@RotugJ3RV{!GW>OO7 zh=?o)5wtPR z4D=5d&fc(L!{kT75&cguG%YN?L~c(uY0Z_#1JsvzFdja^ulf3?MAe+Ju;*E7c{cG1 ziH~;f+BNcijIN7EmsNJjpgvwc^$O>Hjl%p^@3MDc4|a*hi+Z!h%f>?o-VfM(vZiNvb(duXnx`^Ht6;v~Dav{4mF|r+;>G zpualucBD9mPo`;WJpO;2VzT;vrPQca%ME)jNi`;^)#G!Hslz>CjuK~XudmO(-YNI= z>|Mvy^^}=2F3hrw+^YMq9-+sspB9<_cJs!K8^gI_H+CLBt~}6?Slf`OVtK^XV>8#p zbh~SLQ9o(D6fGPdab2Pgk4lOxL}eH?>gty!Ry^jH^cGpnR#jEqrgqV5 z_wLo6irsm# zUh&@j`*r#DENv7tD{9kByH%tT*(JiHT`WH;8P_vu(FnDD`G*#H*z( ztkA0r6EbWsENE$I>FV!~FJ*PSB(gB^aA?MZktbg2$?4gSn+(P#qobo178WwWhjg@= z+!GgWF{=AyWo5OFWtp`n*$g$mZ*5h>3doiRaAoL}Z_X2j#rc<9s~#y@N(vKlgaOGo zbUhGFm3K82yL`DGFV1adX10k(U6pKxMCft!TUM{YXAdmO1Z^^M7Y6y3UFY(vvVUwlXbF^eSIe%AkNP^44ti|=~(oLy4^+{@i*t%MU7`*2E z%=lr>rLPWVeB48=d5Q%NlQRDM<-4%ec+1J*Ru$L989n#zFkxq9=h+GAnXzu23}v#_ z1>at;xhN)9ouXB$opNQO&2dC0$P88uCsfY28|Bg{!b~=;*Mxm)s zTE{KAF1Wimv31N*KHc=1tFx>WikTVjIdI~H>dlqwYDPOtCqLh2GlW$%r|Njz+sN_M zvgc!*2{m)SO6GHS2DT)ua{rsV8)nDKd3KBGln2y*2tGXddJRwS-MwK_f!u2BV(YdX z$xdkg^5x6Kw_=6?5}uCe`;_@{M6Ot~xp%abt#PcoeDci($qT+;KLkf&d*X~5leV0& zdy%AG5MSUpZ8$eI)ZdgA!`?U36HY$NXWntLK2BB?%NTRitnK;2-1NZj^sMhv-6x7$ zOzZcy;vJ*ERy;1RsJNV5F!hK$wYTu>g+%3y8W>pY%aX&dBdJhE0!KVd-kmF^{q8ot-@j!wzi57cAhh(j=((wU>z?si>-!R z1TH^5dim6;Q}?{Q6yWG-1wv}MR@G0=+)lR}y;d;Y=7iMyY3v)0~<>St?=>a?% zNe2{@VlVpdH*9zwyqQ}ycE=faZ9BD#e*2yvGUzB#`)Xr0ZP@Uz+s?J?-LKX|&GN*F zu3QQF`Y|*?y}*HE`}UVdP4#qPOk0kbM?B&;OTry`VWa&mIcd1>v(=mSYxjy=|8iD9 zK)|54>dD0m7pkJf9!`8XT&WLR%M_-$!=lLYcRefDGF_b?m$b)5yR&P3etpYkK_OY) zq_OZ?$?7w{qvMq#dO3sRJ(XL1?;q59tDO)qInapTu_HQ7J>7vCN71%* zqu4~xNKx``L3v@pG{nD>5q`jJS7$^P@^<}V%u1n54Vh7X?aF7x+1YSR3v`R#pXtqX@7 zCVC$vnw)T%&)&hIQj08Ymh*gKNs*0>ZP&qr)eVbqy{*0$=|Urgb4feiA9I8 zoVr_N@n8rE_u(a5Ex&$9$q<_VR(w82B6M)JB2=)^NBdIfalYb1H6q@w-#^oz@ja+t z6~Q?d<;?<@EU)VM+Kk|;Gu4y_Z#mGmy z9Z3<~MxYu1?Mw zn@kKFN%fi$k781Y!_;6LG6Qw}L&LWG9IUa`w~kX{{8N5e`qiiLKZvWcw|+`6WpX!R zaImsUU(Y7|99g-)Qe-K8K?4}5FQt@~M8?CaiPPBk%7&v3j*dn{%^6LP<`)*yGCovQ zN#%LOTEX>TMYX`T7WqO=I!x|`-@mN6^~WFYc-Q6?4Rv%}-ObLfeJo=KC+CML@x=;Q z!tTDeajPgkn?$|i!~-}}LJvGn0-&57&KntApR)ADgQx5B=O=;c4jkC|X$9BO5=NxH zEr)e41vp#mOuOC}9!|2pJj`rp+qa3zepoi}tIUoaJA#J`B$MM*vP_W;t6@|om5EAe z=YdnQ3KUMx?wITgpW#oCjqz%gHi2`>S3Kgg2E5=5#QsYx;3fMTlfzdtF1EkQt~@b* zDPZ7d28JxE(c{986O-SZx6geSHec%e#kR$kEY=@?`~)DOJ3Ehy*~Pe2GV2~n3X{m! zhrlh1NLH6)sW&;;B&(1x8CGvQ)+M%Lng3(BNHPhHx)qOt8y*J) zM9aoV#F@6{X1=mPcmg=PF*Vo}nbqbXP7(s@29wI1oSdeWFN<$8i02+O*WSHj$1Bs9 z$#B$6L)+`u-%on1!o#!UE`ts&yF#&4W;BnW*d=i$M()5gQ%it-oC8~sN zLSxzYrQ6HfDgtFF(<0cxrD@ltm_3&t^%BFmRdL>P3&-x==YaXrSY{n3dZp{1E7fxI zwhaO=krLu!Aa`h|!=Z;wdPBF?Fdi{&ImV@w%3iEzdHVE?sIB1xckkXs25sXC)JWA9 zgB2mnopV4=+TD6AzqPbf6d7DU)9^^@v7hpvgGWsphDI{25q)>`eE0ftXgbQdUf^n8BB|hi#*ea+Oy!&O+ano%-5~NN`NsM z{5WJu{%W-u87ojg5yNuAPHJgk#5KcV($E=p_~@XP`+09xlf289vrJn7_$?E!q?~Sd zT`FjEo>nUu|0qZh5Xq06fojp0nAM70)PR1^83wg`Nc78`ONv=?n1?gOYZki)b5DO=y8YaRu29L)Fdlm!o>=%yJn*AG zwX{pyaoDP9X>qzJZQD09;lOc)Wck^^rxW9?l(SwADvK^ z_9hwR^t6z@X_d_|7z3Cd0oL^|G|%MZ=I#e7gmafyhtZq-XmPPd|9LfT? z5?~lQO34o&K6G++HZdW=4vS8ZHJ|Y(sofX-e2R*ST3TDlrVh0gP^UU?OcZ&9X6NK| z&n8ome9ESh2L>9GD;tFDYh8bp)MIk@+9q(#aovcvqv>Rpf0HGN9e01*?f2->qXWtH zK*!ymKWoletfK^9@1lXKO&l_3!=0y;+}*P5$90{*y`sN&_pUTb40$Y%NqeCh3f%Is zof`593iXfpjI01DIH#Y+j8p^PKUH&_Rqq`9j(TY0rcKWfPz)w|t23t_m-?jYl$Qa` zA`RpX9inC!#o#Xtaqa50YwKWJ6DYvjYzkeyrgL^LIF_MMugkSIQker9vHtSr?l!ec zfg{VCT)jq?Pm6p!Rq7mr9lm(^at6!FDM_HWtaZ7r3zLfO8h2`4yOI9_*f$+exFqxT z_gPb`-l|iAHiK1}CRAe+lwOa795d(V=jCD~Sj5WA4{X-c(<4yaU9%UNk#@^HfgfvY zWxLA*v+BeBk!vpc4rLxv;*g8=Y0flBNG_Z$v-uQY)!F|9RcXyG?-M)u`T12Gk=F5~ zKXa+&*4ezv!w#~#eAm1jdR!jsl_uE8wSD_ZvSf&A)~J`(4pa{Fv<2rd_JlYCPUc$m z`3{YuGO(T+RHAzCuuT=}n>MN#G)sV8F*n0zX+^w-)9ATqjjzKK8F<#fp%Drt^~<}1 z)>uZ8DT(bNEjM58%Zk2ORPu?$^zYy8oWFm*%^dJhy%GP-QOrB&vac}5`|;z)Cr+G5 z2kgk}(fbheAo0<&r6I1unaQA|CXZURQ1e?4w=nkN9yRJ1ft;;oie}b5CKc8g;{HrIb<+1d;R&uZ_aKb%fGWt!M|!# zb;Joz0KP#ES(_~8&U5#AzkMzy$fV76$(dl`HJdrB?lT{rKs^z0c8M2Gp@c` z#vU(U(LN}W>6v+;+!5#(dq zdVFDiyzbX zL6_ltY`JARaLpx14pd;<*0GXfB*gWN5`%|ZAjKgMiEIaweGL0#uKMIPMae5aZ|G% z2-psXObxdt&@Ert)jzPjW3e;#pf$m77F~B%uHP19pe8IV?9lN$Jz8+#DG7S@b zkrFZcx|T^xw)E}wEM-$hU%!5hM%>j0`Iu`n^vKD{X<}izT`!l>@%0}+&w{5xfVev{3pP~Th4VK%34zmwOrZL{A?owVw&6FZ>r+((ls$3i z7fS+gni!IDES@wPkD$LeKNAnC&#U3pG+iM>2iw?~@94_{^KmZCnu;(X>Z(<%c<*iI zG9)ETtVU7cXtPl=v);!L(6qcZBW)%Kb;TfRo@!<&ualIN?9^DA4NL(#%j-67&8;U_ zqY^T6bPSrHDrxgQYgZluusDzK;0b`dg`1oEJkZpAk`-;rXHlHr$J;Z(sSZ9@NC3Q% z@2pqU;QQ^j-_{mqUw`~iQNy|Z+w?R)%1X_RsAI_Qy2O(kP2eEz2?<8-R8#Ft&pg|a zqc?8eT&Jk07zq~XJW{78Hd;4VRZ_BYZPvi2eEYN=@AcdA?W4yBo6<^qYo2+Ss2q@w z*R~pHsJL_QUh1S+QDPY=m9z}!9FB!0>(MVIZ}0rNYc1X}5*X(^m^{yidnI*y!$XI# z7IuI<>ri$TqskcGd;s~8S2osYJyhkthVF)dR2B_}6K!%T~*RI1%zO6h%U zpu>8KXDwAxL#+InIenLmE~cpl7_=hP5&Yo5(e>Lk+ey=qi?#jUNa z`=p=#QJtv5-wC!&sZEy)*c{9XZak?l34=e}-M1in z@{!$mc5xGV2b-+j;)+?OYUEYFquxn%QZvnToE~1AWy^PeYnIZ$rxY#sMF5fy6AA|W z?KcK*G6?hT$I@O&REPxe^%jimeK$8sBti}I*8=+!W&JibwAy`b@inF#r%kIUGpB(r zQ5rqP$K${idU4a+$4W<@M`-j1y(T0YxQvp$+;a;biE^7ZZL$^}C#)F3tHN`GX^wJD zo*sb&s}Xhv(RsMZ!N>8t@KuSBo9j19p~!O>%4nPbWc&JEJ>;D$xPe5VZKF4Le)ZyB zK{2Pd8E%Q|?bW~K?%p<<4}QA}eCFka-U^Bx;p|)KJxJI>qkmwj9#F6Xwv*;m;v^9@g0adip8y8A7O9jzz+u zh=?ycuj3YOV(2WWc9wc8B21HTe*3ni7~y_lIKtmMmQzgzPM2ByficIV!;ArFuSwnh8{*=dcm+soiDna$#ZN7Cu8i5DNZ#CznyK+F&Vg zEjtH?*y7^iBVlKU%vm<%3X%@sEt^yRSGBQ8=XA3QCqtUEJ*fh9>JyMPznriSlhW^#Wb9>^W% zG&5#lQ5H`B5|kw2(`-z@o;%ER2LkO12q=+Kz0|!{EgDO(Qqs*PtLK zG5+us!A|~L@;~@TwPr+2Om1e<~pB(D9IutLZRpuJ7dGny4c#< zqW(7*-ip^|N-*uP#y2#zddbqZ{;##sURu#P!z5S#{4StqY(C zlD4(;BCW~yPAGa!?+`Uh5qUbT)fK)u3u2lqmOr*~L|+YAPQtL6mlP6|g5 z6Aq{;TgVB zFJACggo|jP=ag~FS`3s{Jp$QejW`eSv;C+z3A6w{JWy#w??b;?X`33IkF(G^QAONw z>{mP)NaOdssf(u%yDrYJH35fFgBy%!%G18FMI~DB4Xw@VcH_?fa98(#q&58i@=f3` zTlr1pE?l@%7RbYc-U+HuUB@;g>sYXphM!)17D`|s+u-K-5=Y0_izN}{|h(9LygLyG}*$f8$|BcfM5 zEDce**J~>GN7sHOc35aHT0JpnH5i~d)6^f1;PB?n*}R`=HXvFyO}Bu0W{a>>_TtFW zqU5bLdLwRb@7`S?4zfo>quo(;H|>*I0m;i9z_bwh6xJ69k8TPxLn23a;KgNd1G%^5 zmyu7JJ4OrAFG#4OBO+)!TnFa#2>MZCGRn%2))q^ln4E+Y44_F@!5fY~YrSE`fW?m> zBU8A(*VNQ>7PQJxGl{-zF5geVxfH>ovdA&h9>eyn72N_VSyC+wje=kx0i85?L2Ya~ zc09zK;Iv*Am2Fmk&)eqG|a4>ho2wKR1sF^}uckcMG-@0|n0Lc8f zYm2MT9aBc20wdz@DTf3WsBg#vqv-iO_g-ZAn*(RBq3e(&V0BIf#w} zf9)b+y}-cM!N_K2W=M|{N`+FBY1o>S6j6h`WD!Sm797FGjT^O%EYSd$lJY?<9ME?H z_3sNbzIE1gk;Xh2@jA5fI2%c(`eBT2bMl%r5z1INNQGz7(Pd~^C$*w&4TS&BDhy_{ zUszb(`^4CtwZ&DaEGJPgw~zfY4O8{p%FnH6gqL_ugK1-LfX0qMx=|5y@C-qKq+t!Z zOaXlj-GFg|I{-N{`|9Fk?XMi9Q0zT~@a`R6_I7r$=-{LFO*8WL@OYS&Wg9)7Kl*m7 z!&H-AeU_Pqm<)&se?Pw{^y7?B3IkYWx4$&ZC^O9Xx(I#H)zd#<+lQtmB|18~dnj%) zkVHB5@4pBnQehPac3OMY=Ub~c`LZ*+Umv-*g0emQDg$ln?%Y`gOC+8}ntq^#q9LES zb^ErsFHmTAXJ;hdQ%nZogzOM}kAs(2O4Msx^;Wa|Cq6#Es7TRMBDU{>SjTN!ABaN2 zg;z(49yoeb0VOm6izbX-9(V7iP_=~7LPKK|nEqmnWVixKm`%U^_B5;Ag)03R;VACP zS}zXS|FqT9KrSx-pOOXrgEzpL{)^h8zkCy3wUh1H*CSC3$rU=;V|9q?Ct#syeoP$f z^Lf-|{-CrT9W{yPGivC%{3nw74H*`@G&476G&|AP)L(?mPCkwfM-9AyP>h79hc;k3 z8$vStSV+Ri&C^gaCIIg@4U3pGAIU2vbr0(8XRuG~EMHl7KkdWW{FfN3MidY?p{j^N zjMHoS8)Qc_z+QR_?PE%Z2;edr>w2H-G?NNM5TFzO5AplNbDGG@?f!6rTcLjSPB@E1 zxa(rh2_d1hLH>V4Nt*t&`)OOXZ~IcH{r)#XouFGGDIp+!AgV8mh zh}jPoWe9y72lVGQF)_t{`gBE126aR?ibfw3+O0Yy2sHc?Zd18xcJAC+2jK0=Dx~ad z1%O)e=FO*BtgZ@au)EPMN2xAAJ%Voh?K)a&Jt+}n%HV+A&}KRxP+1u+(mp& =0< zXt(d8s~sX`76qQO^M70B_LoiZe_e3*f8|Ns`j*Y2wLl^Zl6CpD6n*6ItWZ7F=Y-nR zhMou9BEaoZSR;@~)IZ60OJ7Pnw(wqcLo^^a1^YKp+?JVvA0yq; z*Tu!VTbUjfEljB(e=36ZsH}1wZpk5~Hj(y$oW9_TR>4@RHidGj!pgB~qz=iFw6IA1ZNr8kR3tvmcv3>lw=xYvbr%s4!CBgoT(qc++)@wg@9OS;hAPGY z^dQ}&z4cl+wXXLD;gRa zMr)ZH8yiQDZxgh+M4%TCABu=){B3q;WTP)ttC$t|cRPHSW}!&=bj`ViZP<1Gz7!}V zljGL?ev#tKC>@1?0Rel!-GS0nK@ucgE6`|hN@=<}0t_N35?A};p$ZBLHZd~lHEw2P z+zVW_iA(uNtv#qOl=p+QcV%T|MVTcPt@VPIhW!pGN}%9IdcI|B0{TV4*!a1OjLf0X z+idl8loeuy482$ec{w>hH7eGq9xMR8{7;mZ3za`|v5l5F;^@<-PgA$ue9UJQy-U<< z9~W0egH3PM4j^+0ka@u$i@p0PZa3ci0GK3dp^Xr=HA~t9zh1g@KWb}H)NOb7=2?-Y z`EgROsyhzxu^cl$d06*D6$(`%qdE4}i9%U@AKAPj_^>ExHUKLSxz|1pj`ue5(`XFwfjX*4mqfG}R&q&I>N)~gwn#-);LOzxk1cW&c?U<3rZ=s?=v$7^WR6&7Q7CjFQsD-Q68`MAsm^8%gi%ERzjDf|K4}^MF|NTD`3QL|C{e#u%+bMd5pD8C_ouxgJJ$e`Q z(U-S^ot6b+SAZn$&_o;*QDH8#2J~`x_E$fiT>yp$j@PPO7!u`12<^sV@u)st9&hL< zQf?uFwi47Vv&t%jKN}*rq}xPCakKidgfOmNfL`WiA_^sz7D2m`_Pk@TA(}|AstqZ2 z0X)t6*i{J31X4=5flvlHq@yo_Drq(2*P0^~%CXw*0_ z_TmsYEXi&PW#e|R_X?;#plaRygEV7d;R1Q+3|PHBX+2!dllBPUaU+SG5hda*q+vv! z2>_e`0Zee)2EyO5kSb@zB{T@?i(Fj^F-b8!OiQahL6zH6mQ{=T(*Yu(; z^@Vx^kegkMv}vE7^B}ERmuILJ*pV@!_lDV>E?S&T;N0x7w%CUq&`(STF>hD-a!SO> z@{EM+Y3NkY@(yGteuNl9%!>pxl%72ehImIcCP*=fHxl6|S|CKqM>?NGSiJS9vALB! z)<8dJbJg?^mB>LZ_zEx0TSCq*iGr#g5KQ&_-9XF@} z$`e7^x#j(`ZJY6&Ev~ZacuA+0ZGMV;M(sU};n?}4p;JpRTK|v`<25~dCBvW^sU=b_ zNRPgfjXAjOTMAzoYz7&A`51QnCoG^X+I8VBvvKTwz!;>!hPs7llHh!YvlKv` zSj1jM8#p)(gB- z3;09Czxu#3q_e@^2QJsb((=y*s=`x?1_HvS`WVqUgM}tSVE47mPthTk!v_0XX_+}h zp7*r)VI1guzpVzJsNF#8f+M>#0b?y`r=(=_3)F;17itfhbG(J z(QyE=&WD}!tx40l0t}*CUPo7#Rk=$1^;@>s?!Ar69FSt$90s@u;~mJO5y0-hPx^Fn z{@6V%569jZ)x4`Gz*0PccH)yzA$qPvjs=ageoOZC_3^7IlnXgbRj6s>B!y?rlAbNd z26nN>{ALa4b6G=cUI2BhZF)Yi;y|lyyF4HO7_&*}`FjE!J7Z8X2Orkkzh}>RV26IZ z8)h{yK476pT>%^r1s<5W8J*sER7i#>NBV1HyqCv=v6A6lIYE@xNS+P&$JEHED&lw2 z4n}vmP5Pupu2mcgk0VA6oWvPG!mM7uz5%(iyRR?!$r1v}5r~5c!Xq5VdY0o+(Ad5Q z-~|lKNp*%dRT3#OI?3(TIuwdjM0fM`9@vQ##Epm28{_50@gLcdIz-C>dO_CCE*n=y z0xywZ@<`CuG1GIRW!BB7f39d()fF`nWGS4t@@-um5qIQk^GgO{w*?aj>6k;kMi*=@B zgc%x62){KTdr}KR{yM2M0Q#>o-Bnwgo$)mtW#fgBg6XhU`IgX-5HjO|Js{%|>z|_# zg$^eQyE{NV|dIGdY?F2NQnVoG*n+5ptz!nPR0tWfqQY~O`fS~5E>e^uh#A?z^B%Nd< z<{tpn2l~;$=FTr)x`u{A`%akUkGc~j0x6w+<%(RK$;6E7(qeGimkJtBq-&c7$1?i> zWRR4UMC9ay&4HMk0W-7@GEn09l-kyimVd(_mD%*pS72hucgG)$`{ge9YBRkpIIx-G z=CtR~AsIXibF+|;5Fbb{dO19IO?N-*M9XhzPN9WB5)quX8u8H&8s&R8zEndAK7i{xz zFTp$_LBmIZ!8_XQ$KLNH!)4m~;YSQ)+#Mr9&gGw@;gu$kO~k{V6o3Y5#(yl|k4V%< zIM|KbN$5;Vvvi+Ag@}2D>VJfA5T27KVr~xPRCrV*x1N;%LvLE^iFIJ&=690XP zO75J;W&;@TKLRaaHPVhSgkLBZI^_^T8I|+`rca3A>4!hlaNkR-!B5iYQ~R+a3K#-+ zkscms+-J~L(<>58T0$0J<4Mx$z=Q!wWGIY#LGIUsgzxH2C5VNPvM1?&-MTda70Cc; zFoInqohw8a@NB+ofLyOGBh+aGWffYPq#tiPqK@o!nRMP<7suH|16NQei@y_|b|aFWTjlfSw{@T!_+qTET z&be?E3`aeMqKbrkm2v2T&tlLEWkO-rie(fZZ$iGG^x{;@l|$Q*6g1EaabZhW(9;h9 zE@Xv_V7i0s9fm!^ME!Az$d5}fMDXt^^ED;zJGbq~Dj* z>-K6SC~z%csKLers0y5H-~`~E9&>J@Lc((eqohXI1M{!6?Kpe(EJFGv@U03EUL66` zi)26=E53*}`_Y0vcm8}?Ny#}h)6*UoK@{Zc_XHKyjeDdOYY_#V57C)JQ=JHEl<0FY zbC;=hj2i$)LPB+TCBzT{QP`5 z3?Ex(bXZXRYX{jZouma6K}Vg=|f6G zlE$GSHe}d;;n@bLv^9Sjif%*YU5sX+QZWPuhJ;dTdbpJjlZFU=2Wl7pfm4%XRs6xY z3)P0+9LDg+>07HgxVb+j32}2vU~VkJ!@(vhzd#)omD+#0VzQb z`j+LgpF4Y2ADSRCTS^qIB>F-`l9`YTPRtN)l)1>0|Lq2{bOgZhK-H5ovzTQgh-4iT z?@Q8J!p}>VgJ@Rx))hF~hKjhVV{D5Md0UPe$0CqZxwkQH*syDF%+f+;$G|`WS}pqT ze0CWAfa8QI0}lA`w>zc$SB<~S?{+qAL8Pkwhi2IS^S(r)41WGTO}`O?;^c3J4|{N; zxu1QQ$T+s&B^eJt{}<=+{|DaXZ>w6n67cU|HL_Fr+M!2T*C?(13Np=aokenwlEuJy7jC*ovd z;x&p~Rx>atLZ?P5H6qUk9aCWuhNL4j{^9Tze$x zb4W-Ch<8F}=?JtF2Bxns4h&z54GR=<7>!Q3m;j)Vc$cMT{0!Po1Al71r(=caN1S=03Ezx8(JSvDaDxxEr0 zQfLGxasjgV%4M5tl0XXaf5zidT8+_mv!Aw!@0iiwvbC{MfOH$ox3LLye&|WdU>NG~ z1epwR!A!$BQBi6%7&BEUBEWYaxt&aV+_4q7HZbb1bft5$vtv;&*^C2{%J?2ir(J(l z;r~!2SXdlt{pdKoen)6G=EvG2G;ewpuk}Ljf z-BSZE?L6oI7~PV6+rr>OAu?KP742q0f9cSPe=zcYqP0jJ6a+xzGiy^NH809tn1)sP zBCzKX;f zWfsa`e+;w5#>SpV+D4&Rl2)bLzX!oo%Cn(DMkOEpw|+5N#Wv_DOdlYSGg}MFI%Fz; z%q6Jh;g4%tT4@>g(H|g6QJWo0x(^;az({r$k+Is!^FyvkXlIJODaK@!Q?>6ZJioGo zkui{SjPx)D)OV&_fa53on=7Hy!tjL1E*OhYl>ZeCtU zQ*cfu4XmJS7omwDvbE5KK{9-i_flncMLW9fkGh=zooXJN6~-YyBtRTtFTEe%u#(`t zlU{!XVO{iXv+rRcEn_gyzQ8U+s@Bk|W321Au^<4@njgv;$YlmFvAspeAss712EP*3 z3z}^{XG0(=Ah0lUwSaI-7~ccz%~1ImbWrPv&EThePLzqVv<=y|6|>Z2e4)=0r0)wy zC%mhp23qekOg;a<=qvEO z^`mn}tqOQuh4ms6AW%viICks`z*>3w670JUELNgMkvfnVkzJFK9U#n}>)kY0?z+#h ze3md1$r>W05rFA-2&4n1@tw4~%cl{yzNE)jM@2^9(U)Y8)f^61SGCxk9~v z(NpmGoVVyK@8*Ck3}Oqzmh6;AM+}|<@M2lSXjmF8NhoypfkyJ3AV=YVM8-hH0~DRc zJ@I+Dxn=eB^_oAR>q(p7LlL$kZE84$rxDX7n^4nZ8x2vyWUeWN9OK>lK3BmdRPAwO zN^z*U-c*2y(ZDH=r~W`Z7H|CM)O_J6v80@uZNcS(tl`J5e`#X`Zv})Xq@8OC@5ax* zJO5Sv94`LfRt)_=Dbu*x>;_O{1@+#06rLE?LNlwBYmNsat?04%;tUrE7k~^Dx(U46 zsCvH;e7p2j-w4-fcYEfPwm_n0r(W5gI%0!Uute9tT`!rQJCZo%|Zl$nm-9{ zgqiK`vZGfxb5fbMO9qX|c5L1}`t`JM&Ee@b)#+=`b5-nx)Wvg_j`OPT64A_GIs8O* zOWKG{$o`N6@?B5=ECYeMUSyyl0|w&)f={_s7?2-)e8FVNw_pv7e6q^yFlPX8p;8#ttrh7CZc)(4hT0n zYPYrh+`?dI*3E#)Q$PVSuoEf%UiN;|3`01_%Yx#$@_Ec!9926x{clN=OFg4gQ-zU> zvdr(lM^YDsnTRYcaYU};R=)8M* zs63Y~TL#8N$4FdUTo#%PlFIu|pv2D@;33DPRHR}KB3qDVUc!GFXo}T9n?t7ZNPh|3 zLmldfbPT|e;kgHoAwPBY{9$WlZaDGipB9*$IPZ%Tm_ze8HRCo&q-m2qkQPdy(t#=Eut+ z5D7%T8&t>P6Iqy8L_Zte-tDZJ&7;ky{{h1UyTt?YBXzo`UfM%J2R9XN3<#9{asHGv z#@G&>n8vhL*|n5dgaLaECLe}_x`%al=`kgnd5<=?T(W=6*RquiBrG^0w-dYsE=}La z(%L#>fy%oU!d;xcL0!7*$E9-Ac1^(=B2qxMPB)<*CWKQAQG1i}kDOyLv8Q-xnB|o5kx~cAF=jce? z16~EMN9}}b#iHX)PRZE#%P-rZ6(gD~q$i}?hv0O`IbHN@PEz28(o}{kinx_IHc5_L1qcu)hko6&j#Dlb|5)7{@s(!LM|-0pdMX9at15_4_wX(xZT&8xyNu=(Ro8 zWJY&P2CY3goaaEsXi&S>VDD4qd4@&$Zi^BafaQ?IA!0|suaRbUji<&GW(DIQH2O64 zhuG4b&Jz0V3b+u0I--Aouv5vTIsE|;I7I8jSI8K@Y3R}kZSDo1RM?*ksq=93Sr@v4 zhtsQI@Y4cGCG1BFNLxWAxEz z^Yc}`FAk%9dO75n^xX7_B9us0kdBg39vrUJoTldufEELNGq6RAg#ZSMWU*lB=njyN z;3Sg!;G4FK<>G7rR_6(6qAnpb2ar2MT^56*9Gbqm;vk1f><^AKE384EG#Mn4_IIE1 zi)ge@;$RbU;FhnclTW5mqXgmg7Es|lAqLDikL*Q zg8!*B@Paj5$8uZ_!l!uYNOv+tk0%xPCEtclO&{E?4$&~fs!s;f%S!N-^osTbMIJKH zhY!+@3)6bLmJA7~P}?zmLY=zE@8aqDHCV4}aL~AM1j?F5$Rm!}k7<*`+Q{@Ee8(F0 zELH=(B<&z9GAe+Y6D83VJWHJ^bJNTTx?#opP4}Bflt)_7jyOqr51&EXJLcn{ADl@< z8&v~(Q0o*~n(-dGj#~RZ&h@ICy5f5f5_M`PN z-Mbpi?YwT&=JfaG;7>42XGWF*2XCPi%j|)4#_U3w)U=2vR32nr8v;o(Mcs*DdW2h} z(3nh?e}}T_KFoV7CXv!7c>oJWLDN#XIw6NUqK*R~@H(B5-ol5-5l{2j{LCJ}c>|mk zHQ1!Z^ma%@a_z2xCBf}QS3l@}I6rkz{jhqE?ck}~p-}r$g$kiSCk!3DHUX%COsqlw zXf|i$1kW~J!l60YP_eO-XzE5j*p`hkfolo0H#-=BR$$OdyQjShhcGq#pQ&$I#- za{3H8PYyo3A2kcqlJBbYmKGPt@n>o(K!@qBOD@!L@D&8)49&QP?Fua9f{uiY!x~d% za;*&13LNAiBdf3}$S$f&R8b^cxezzb-^z5Uu(7epbR4-LgV|Cfo*0CVa-7bHF(jzz z$5;yJ$-JPL&B!bCE~y3dH*ek~WDKVKjPMAgl?13`Tuzc@CQp(q2F@1%Udi*OBm0t` zda(@|l7+k~%bDS*e$`22fXtr5jtp)6<3DAGljPbt%!^fg{43+#o1xGnQH5C1sUye3 zfig*e7GpmQ4Z;-%>X}zKsqfnKr)w$Dt|A%m{DOm>9C|`+4}HM92taV(#hrE90j@iN zvjYq;%uXg((yTv+gk)o5@T7-?+Dk7netFf3d0bia zK3MJqys01CuL!q9a)F?>R?vYVnrA$Ge_yW6W`p( z5gPz2I>%QjCMwIHySK3#H#1RuFpnIEV^(&M6Btwm_Mt`1>Nnw46eTl4w$7*g=~?Yg7A-C2p}!N*H;!7V29X@1P;R0c^^L+o zMg7>hCUsFrU*r+e5Gna#}i)x)X>XxQOfrx8ct{i$r-UOI3<$>e6Uf& zb7`oCNVNf^Lv*PKb@iyQA|Ngxr^hzsI+uu$K~kja%wj#$Dda*!%)C9qad_msE-Y0) zB=hNNbvscHA&E*M+qgiQ0Vp8`rF6Q=+ryP7c0o{xKre>#`pB`D8NfR@x$*+q-J&1W zw+Gd%;mMCc7&U+-gXlX*ixoE}ssV>?f5l;aC7eA)qz;(SbEdw^gT#;=i%XoB9Q#9Z zIk1KU|Kq11BgqL&23l$zUVy!iVmWm<(n`bkKMJpF{+%MCPo+R={{$LwH;XCKN!*yu|&uEPt8*=Q;d7z7(=B%=nXLz-7k z;Dkv<08d4LCg`gs-tF@K6sqx%1sO>m5H8iJ5Df`3?81jkT8|*a<~;F$OlE@P74t>^ zfe4I=IG@+IzJNl2 zxCJ@y1`*g+w0w(*OFkKFfZeuDy(lBXIviva$ZFR?N6;-fXcTimBf7F6nwBpBau87p zq4R*;$V|IUkLgE9m5}GjBoI+CLdUk`)u@8gm{h{(`EfKT+Je1+O41;_X=i~gOg0iZ zOw`tC`^B%1%_iWyaIf=Q1T3`O&vd4a+b`R=kpP3M`MUWZmpGAGy|`(lo7)Yq@B_4} p>c4xW^gposxZi(^3Hp7B{`8l;yK6aByvd`9os&45bo%Oz{|{^s`bhu) literal 0 HcmV?d00001