{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "\n", "
\n", "\n", "# CSCI 3240U --- Computer Vision I\n", "\n", "## Lab 10 --- Convolutional Networks and Pretrained Features\n", "\n", "Faisal Z. Qureshi \n", "Faculty of Science, Ontario Tech University \n", "Oshawa ON Canada \n", "\n", "\n", "*Fall 2026*\n" ] }, { "cell_type": "markdown", "id": "35907a9d-a37e-4972-b052-999a018e8459", "metadata": {}, "source": [ "## Copyright information\n", "\n", "© Faisal Qureshi" ] }, { "cell_type": "markdown", "id": "57272567-9ecd-46db-8e4a-728ab3238c45", "metadata": {}, "source": [ "## License\n", "\n", "\"Creative
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License." ] }, { "cell_type": "markdown", "id": "bb59f095-bc5b-4da4-a108-62cbe53c596e", "metadata": {}, "source": [ "## Tasks\n", "\n", "The goal of this work is to use a pre-trained ResNet model to perform \"custom computer vision tasks.\"\n", "\n", "Specifically, we will use a ResNet pretrained on ImageNet to perform classification on CIFAR10 dataset. Recall that the ImageNet model has 1000 classes; where as, the CIFAR10 has only 10 classes. This means that we cannot use ResNet model out of the box. We will replace the classification head in the pretrained model. We will also have to then retrain the classification head.\n", "\n", "1. Download a pretrained ResNet model from `timm`\n", "2. Replace the *classifier head*, i.e., the \"fc\" layer with our own.\n", "3. Setup PyTorch Lightning to train the model. We will only train the \"fc\" layer.\n", "\n", "

\n", "Go to the end of this notebook to see what you need to submit.\n", "

" ] }, { "cell_type": "markdown", "id": "351fc216-ae0d-4c39-a0b8-04fbd190f03b", "metadata": {}, "source": [ "## Get a pretrained ResNet model\n", "\n", "Install `timm`. Check [https://timm.fast.ai/](https://timm.fast.ai/) for information about `timm`: a deep learning library that contains SOTA computer vision models." ] }, { "cell_type": "code", "execution_count": 58, "id": "b73d023c-1e17-4acf-82a1-19a6b29d5544", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", "\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", "\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n" ] } ], "source": [ "!pip install --quiet timm\n", "!pip install --quiet jupyterlab-widgets\n", "!pip install --quiet ipywidgets" ] }, { "cell_type": "code", "execution_count": 59, "id": "aecc44d7-46ec-49ff-911e-2a6fd0335b3d", "metadata": {}, "outputs": [], "source": [ "import timm\n", "import torch" ] }, { "cell_type": "markdown", "id": "8705a164-ec64-4d35-9aca-01b47801c949", "metadata": {}, "source": [ "### Models available in `timm`\n", "\n", "`timm` stands for Pytorch Image Models and it contains tons of computer vision deep learning models" ] }, { "cell_type": "code", "execution_count": 60, "id": "59d75290-711e-47b2-a52b-8ca5a7ba9476", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(1298,\n", " ['bat_resnext26ts.ch_in1k',\n", " 'beit_base_patch16_224.in22k_ft_in22k',\n", " 'beit_base_patch16_224.in22k_ft_in22k_in1k',\n", " 'beit_base_patch16_384.in22k_ft_in22k_in1k',\n", " 'beit_large_patch16_224.in22k_ft_in22k',\n", " 'beit_large_patch16_224.in22k_ft_in22k_in1k',\n", " 'beit_large_patch16_384.in22k_ft_in22k_in1k',\n", " 'beit_large_patch16_512.in22k_ft_in22k_in1k',\n", " 'beitv2_base_patch16_224.in1k_ft_in1k',\n", " 'beitv2_base_patch16_224.in1k_ft_in22k'])" ] }, "execution_count": 60, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# set the following to true to find\n", "# the list of pretrained models only\n", "pretrained = True\n", "\n", "avail_models = timm.list_models(pretrained=pretrained)\n", "len(avail_models), avail_models[:10]" ] }, { "cell_type": "markdown", "id": "98cccf11-af99-4b2c-a19d-dda231600ffc", "metadata": {}, "source": [ "### Searching models\n", "\n", "You can also search models as follows" ] }, { "cell_type": "code", "execution_count": 61, "id": "40750e7b-243d-4e59-9013-a52c86bdc011", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['resnet10t.c3_in1k',\n", " 'resnet14t.c3_in1k',\n", " 'resnet18.a1_in1k',\n", " 'resnet18.a2_in1k',\n", " 'resnet18.a3_in1k',\n", " 'resnet18.fb_ssl_yfcc100m_ft_in1k',\n", " 'resnet18.fb_swsl_ig1b_ft_in1k',\n", " 'resnet18.gluon_in1k',\n", " 'resnet18.tv_in1k',\n", " 'resnet18d.ra2_in1k']" ] }, "execution_count": 61, "metadata": {}, "output_type": "execute_result" } ], "source": [ "timm.list_models('resnet*', pretrained=pretrained)[:10]" ] }, { "cell_type": "markdown", "id": "fe4876a1-96c3-4648-9cb6-3254e566cdd7", "metadata": {}, "source": [ "### Getting pretrained ResNet" ] }, { "cell_type": "code", "execution_count": 62, "id": "71ac05c4-e2e8-4352-95b3-ab518af272b8", "metadata": {}, "outputs": [], "source": [ "resnet = timm.create_model('resnet34', pretrained=True)" ] }, { "cell_type": "code", "execution_count": 63, "id": "caa10e28-6e11-4535-badd-3b358c9db0dc", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model information:\n" ] }, { "data": { "text/plain": [ "{'url': 'https://github.com/huggingface/pytorch-image-models/releases/download/v0.1-rsb-weights/resnet34_a1_0-46f8f793.pth',\n", " 'hf_hub_id': 'timm/resnet34.a1_in1k',\n", " 'architecture': 'resnet34',\n", " 'tag': 'a1_in1k',\n", " 'custom_load': False,\n", " 'input_size': (3, 224, 224),\n", " 'test_input_size': (3, 288, 288),\n", " 'fixed_input_size': False,\n", " 'interpolation': 'bicubic',\n", " 'crop_pct': 0.95,\n", " 'test_crop_pct': 1.0,\n", " 'crop_mode': 'center',\n", " 'mean': (0.485, 0.456, 0.406),\n", " 'std': (0.229, 0.224, 0.225),\n", " 'num_classes': 1000,\n", " 'pool_size': (7, 7),\n", " 'first_conv': 'conv1',\n", " 'classifier': 'fc',\n", " 'origin_url': 'https://github.com/huggingface/pytorch-image-models',\n", " 'paper_ids': 'arXiv:2110.00476'}" ] }, "execution_count": 63, "metadata": {}, "output_type": "execute_result" } ], "source": [ "print('Model information:')\n", "\n", "# Uncomment the following to print out the model.\n", "# Recall that it is simply a PyTorch model.\n", "#resnet\n", "\n", "# Or better acces the config as follows\n", "resnet.default_cfg" ] }, { "cell_type": "markdown", "id": "f0841798-d0cd-4edd-b380-c71bf2b6ad28", "metadata": {}, "source": [ "### Using ResNet for our task\n", "\n", "This model was trained on ImageNet, which as 1000 classes." ] }, { "cell_type": "code", "execution_count": 64, "id": "398b6baf-890a-4af0-b30f-86561cf74e9f", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Linear(in_features=512, out_features=1000, bias=True)" ] }, "execution_count": 64, "metadata": {}, "output_type": "execute_result" } ], "source": [ "resnet.get_classifier()" ] }, { "cell_type": "markdown", "id": "5ba61fb6-e800-45d6-80d1-0b87e47905f6", "metadata": {}, "source": [ "#### Swapping out the classifier\n", "\n", "We want to use this model for the task of classifying Cifar images. Cifar has 10 classes only. This suggests that we cannot use this model out of the box. \n", "\n", "We will replace the classification head (the fully-connected classifier layer with 1000 outputs)." ] }, { "cell_type": "code", "execution_count": 65, "id": "5a8d429e-a79e-4db6-8b62-eb56d5ab3144", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Linear(in_features=512, out_features=10, bias=True)" ] }, "execution_count": 65, "metadata": {}, "output_type": "execute_result" } ], "source": [ "resnet_cifar = timm.create_model('resnet34', pretrained=True, num_classes=10)\n", "resnet_cifar.get_classifier()" ] }, { "cell_type": "markdown", "id": "b489203f-e7ff-4f05-be9c-6c83d20c9780", "metadata": {}, "source": [ "Let's inspect the model" ] }, { "cell_type": "code", "execution_count": 66, "id": "09e8ad05-3d5c-4bf0-a92a-81feaf717138", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "ResNet(\n", " (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)\n", " (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act1): ReLU(inplace=True)\n", " (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n", " (layer1): Sequential(\n", " (0): BasicBlock(\n", " (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " )\n", " (1): BasicBlock(\n", " (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " )\n", " (2): BasicBlock(\n", " (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " )\n", " )\n", " (layer2): Sequential(\n", " (0): BasicBlock(\n", " (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " (downsample): Sequential(\n", " (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)\n", " (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " )\n", " )\n", " (1): BasicBlock(\n", " (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " )\n", " (2): BasicBlock(\n", " (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " )\n", " (3): BasicBlock(\n", " (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " )\n", " )\n", " (layer3): Sequential(\n", " (0): BasicBlock(\n", " (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " (downsample): Sequential(\n", " (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)\n", " (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " )\n", " )\n", " (1): BasicBlock(\n", " (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " )\n", " (2): BasicBlock(\n", " (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " )\n", " (3): BasicBlock(\n", " (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " )\n", " (4): BasicBlock(\n", " (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " )\n", " (5): BasicBlock(\n", " (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " )\n", " )\n", " (layer4): Sequential(\n", " (0): BasicBlock(\n", " (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " (downsample): Sequential(\n", " (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)\n", " (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " )\n", " )\n", " (1): BasicBlock(\n", " (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " )\n", " (2): BasicBlock(\n", " (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (drop_block): Identity()\n", " (act1): ReLU(inplace=True)\n", " (aa): Identity()\n", " (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", " (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (act2): ReLU(inplace=True)\n", " )\n", " )\n", " (global_pool): SelectAdaptivePool2d(pool_type=avg, flatten=Flatten(start_dim=1, end_dim=-1))\n", " (fc): Linear(in_features=512, out_features=10, bias=True)\n", ")" ] }, "execution_count": 66, "metadata": {}, "output_type": "execute_result" } ], "source": [ "resnet_cifar" ] }, { "cell_type": "markdown", "id": "06efec65-2944-4819-9f20-076dfe869cdc", "metadata": {}, "source": [ "## Training the classifier\n", "\n", "Since we have replaced the classification head in the pretrained model with our own classification layer, we need to train the classification head. " ] }, { "cell_type": "markdown", "id": "ac52fb11-8190-44c3-8949-d38be3513f76", "metadata": {}, "source": [ "### PyTorch Lightning: A Framework for Model Training\n", "\n", "Check out [lightning.ai](https://lightning.ai)." ] }, { "cell_type": "code", "execution_count": 67, "id": "4269b112-1744-4ddc-a537-9dddb0135dfd", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", "\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", "\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n" ] } ], "source": [ "!pip install --quiet lightning\n", "!pip install --quiet seaborn\n", "!pip install --quiet tabulate" ] }, { "cell_type": "code", "execution_count": 68, "id": "9c7e0c13-8f5d-4dcc-b0c2-7f8b9cf03f24", "metadata": {}, "outputs": [], "source": [ "import lightning as L\n", "import torch.nn as nn\n", "import torch.optim as optim\n", "import torch.utils.data as data\n", "import torchvision\n", "from torchvision import transforms\n", "\n", "import matplotlib\n", "import matplotlib.pyplot as plt\n", "import matplotlib_inline.backend_inline\n", "import numpy as np\n", "import seaborn as sns\n", "import tabulate\n", "from IPython.display import HTML, display\n", "from lightning.pytorch.callbacks import LearningRateMonitor, ModelCheckpoint\n", "from PIL import Image\n", "\n", "import os" ] }, { "cell_type": "code", "execution_count": 69, "id": "6366a6b1-7ad2-433d-8cf0-d838c3575197", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Seed set to 42\n" ] } ], "source": [ "L.seed_everything(42)\n", "device = torch.device(\"cuda:0\") if torch.cuda.is_available() else torch.device(\"cpu\")" ] }, { "cell_type": "markdown", "id": "1943d95a-f668-45a1-9925-9d54cb1af3e1", "metadata": {}, "source": [ "### CIFAR10 dataset" ] }, { "cell_type": "code", "execution_count": 70, "id": "b8fb35d9-cc93-43ed-a241-b8a2e0e1caab", "metadata": {}, "outputs": [], "source": [ "DATASET_PATH = './data'\n", "CHECKPOINT_PATH = 'saved_models'" ] }, { "cell_type": "markdown", "id": "dbb191d6-3118-475d-b6de-e0e4643ada9b", "metadata": {}, "source": [ "#### Computing mean and standard deviation" ] }, { "cell_type": "code", "execution_count": 71, "id": "c8587bc5-08a4-4448-bb17-7a9844badd2a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Files already downloaded and verified\n", "Data mean [0.49139968 0.48215841 0.44653091]\n", "Data std [0.24703223 0.24348513 0.26158784]\n" ] } ], "source": [ "train_dataset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True)\n", "\n", "data_mean = (train_dataset.data / 255.0).mean(axis=(0, 1, 2))\n", "data_std = (train_dataset.data / 255.0).std(axis=(0, 1, 2))\n", "print(\"Data mean\", data_mean)\n", "print(\"Data std\", data_std)" ] }, { "cell_type": "markdown", "id": "450777f7-2a85-465c-9f15-d744d6c5f83e", "metadata": {}, "source": [ "#### Transformations\n", "\n", "- Data augmentation for training data" ] }, { "cell_type": "code", "execution_count": 72, "id": "c6833792-c02b-4ccc-abbb-d5dfddc979a7", "metadata": {}, "outputs": [], "source": [ "# For training, we add some augmentation. Networks are too powerful and would overfit.\n", "train_transform = transforms.Compose(\n", " [\n", " transforms.RandomHorizontalFlip(),\n", " transforms.RandomResizedCrop((32, 32), scale=(0.8, 1.0), ratio=(0.9, 1.1)),\n", " transforms.ToTensor(),\n", " transforms.Normalize(data_mean, data_std),\n", " ]\n", ")\n", "\n", "# No data augmentation for testing\n", "test_transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize(data_mean, data_std)])" ] }, { "cell_type": "markdown", "id": "9c6049ae-a457-43d1-8479-459319b6653d", "metadata": {}, "source": [ "#### Datasets: train, validation, and test" ] }, { "cell_type": "code", "execution_count": 73, "id": "6f4d66cd-564c-45d0-b2db-a61f7c35ba20", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Files already downloaded and verified\n", "Files already downloaded and verified\n", "Files already downloaded and verified\n" ] } ], "source": [ "train_dataset = torchvision.datasets.CIFAR10(root=DATASET_PATH, train=True, transform=train_transform, download=True)\n", "\n", "# Note that validation dataset doesn't use augmentations applied to the training dataset\n", "val_dataset = torchvision.datasets.CIFAR10(root=DATASET_PATH, train=True, transform=test_transform, download=True)\n", "\n", "train_set, _ = torch.utils.data.random_split(train_dataset, [45000, 5000])\n", "_, val_set = torch.utils.data.random_split(val_dataset, [45000, 5000])\n", "\n", "test_set = torchvision.datasets.CIFAR10(root=DATASET_PATH, train=False, transform=test_transform, download=True) " ] }, { "cell_type": "markdown", "id": "a7b3edc6-0f28-45bd-b5b9-a0006a56dc3e", "metadata": {}, "source": [ "#### Dataloaders\n", "\n", "Feeding data to our model(s)" ] }, { "cell_type": "code", "execution_count": 74, "id": "577da698-14b5-43d4-a765-6ae9a8630a80", "metadata": {}, "outputs": [], "source": [ "train_loader = data.DataLoader(train_set, batch_size=128, shuffle=True, drop_last=True, pin_memory=True, num_workers=4)\n", "val_loader = data.DataLoader(val_set, batch_size=128, shuffle=False, drop_last=False, num_workers=4)\n", "test_loader = data.DataLoader(test_set, batch_size=128, shuffle=False, drop_last=False, num_workers=4)" ] }, { "cell_type": "markdown", "id": "e140acef-cea7-4b92-a4f9-bbf7d5b1594e", "metadata": {}, "source": [ "#### Displaying some images" ] }, { "cell_type": "code", "execution_count": 75, "id": "4d549958-cebf-496e-a927-e2ed56fcb20e", "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoAAAAFkCAYAAACw8IoqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAACaWklEQVR4nO29ebRlV1XvP/fep+9v39StviptJQTSkhCSQEwIoRFIaJ5oEhB4PBRx/PAp6pNERRGNgCIo+gTUkkgA30NaaRIDhC6kIW31fd3+nnNP3+zm9wej6jHnd4Z7U14NeOZnjIyRtWuevddae6211z3nO7/biaIoIsMwDMMwDKNvcJ/uChiGYRiGYRj/udgG0DAMwzAMo8+wDaBhGIZhGEafYRtAwzAMwzCMPsM2gIZhGIZhGH2GbQANwzAMwzD6DNsAGoZhGIZh9Bm2ATQMwzAMw+gzbANoGIZhGIbRZ9gG0DAMYNOmTXTzzTc/3dX4L8nNN99MmzZterqrYRhGn2MbQOOnig9+8IPkOA5dfPHFT3dVnlaazSbdeuutdPfdd5/yOe6991669dZbqVKprFm9DCMIAvrIRz5CV155JQ0ODlIymaRNmzbRLbfcQvfdd9/JuI9+9KPkOA47duutt5LjOOp/f/mXf8mu8z//5/8kx3HoVa96lVqPgwcPss+7rkuDg4N03XXX0be+9S2In56ept/4jd+gq666ivL5PDmO82Pn17333kvPec5zKJPJ0Pj4OL31rW+ler3+FHvLMJ4+Yk93BQzjqbBz507atGkTffe736W9e/fStm3bnu4qPS00m0267bbbiIjoyiuvPKVz3HvvvXTbbbfRzTffTKVSif3brl27yHXt70PjqdFqtejlL385ffGLX6TnPve59Ju/+Zs0ODhIBw8epE984hP0sY99jA4fPkxTU1M/9jwf+tCHKJfLsWM/+kdfFEX08Y9/nDZt2kT/8i//QrVajfL5vHqu17zmNfTCF76QgiCg3bt30wc/+EG66qqr6Hvf+x6dc845J+N27dpFf/RHf0Tbt2+nc845R90knuDBBx+k5z//+XTmmWfSn/7pn9LRo0fpT/7kT2jPnj30hS98YTVdZRhPO7YBNH5qOHDgAN1777306U9/mt70pjfRzp076Z3vfOfTXa3/kiSTyae7CsZPIb/2a79GX/ziF+m9730vve1tb2P/9s53vpPe+973ruo8N9xwAw0PDz/pv99999109OhR+trXvkbXXnstffrTn6abbrpJjX3Ws55Fr33ta0+WL7/8crruuuvoQx/6EH3wgx88efz888+nxcVFGhwcpE9+8pN04403Pun1f/M3f5MGBgbo7rvvpkKhQEQ/lE284Q1voH/913+la665ZlXtNIynE/sT3/ipYefOnTQwMEDXX3893XDDDbRz506Iufvuu9Wfbk78HPTRj36UHb/zzjvprLPOolQqRTt27KB//ud/Bo3Wic/+yZ/8Cf3FX/wFbdmyhTKZDF1zzTV05MgRiqKIfu/3fo+mpqYonU7TS1/6UlpaWoK6feELX6DLL7+cstks5fN5uv766+nRRx9lMTfffDPlcjk6duwY/ezP/izlcjkaGRmht7/97RQEwcn6jIyMEBHRbbfddvInrltvvZWIiH7wgx/QzTffTFu2bKFUKkXj4+P0ute9jhYXF09e59Zbb6Vf+7VfIyKizZs3nzzHwYMHiUjXAO7fv59uvPFGGhwcpEwmQ5dccgl97nOfU/v/E5/4BL3rXe+iqakpSqVS9PznP5/27t0LfaJx7Ngxet3rXkdjY2OUTCbp7LPPpr/92789+e+tVovOOOMMOuOMM6jVap08vrS0RBMTE3TppZee7KvV9MWJ/nAch3bv3k2vfe1rqVgs0sjICP2v//W/KIoiOnLkCL30pS+lQqFA4+PjdPvtt6vt/qd/+if6zd/8TRofH6dsNksveclL6MiRIyu2OQxDet/73kdnn302pVIpGhsboze96U1ULpdZ3H333UfXXnstDQ8PUzqdps2bN9PrXve6VfXrBz/4QTr77LMpmUzS5OQkveUtb4Gf/6+88krasWMHPfbYY3TVVVdRJpOhdevW0Xve854Vz3/06FH6q7/6K/qZn/kZ2PwREXmeR29/+9tX/PZvNezcuZPOOussuuqqq+jqq69W14In4/LLLycion379rHj+XyeBgcHV/x8tVqlL3/5y/Ta17725OaPiOgXfuEXKJfL0Sc+8YlV18Uwnk7sG0Djp4adO3fSy1/+ckokEvSa17yGPvShD9H3vvc9uvDCC0/pfJ/73OfoVa96FZ1zzjn0h3/4h1Qul+n1r389rVu37kmv3+126Zd/+ZdpaWmJ3vOe99ArX/lKet7znkd33303/fqv/zrt3buX/vzP/5ze/va3s03L3//939NNN91E1157Lf3RH/0RNZtN+tCHPkTPec5z6IEHHmAbziAI6Nprr6WLL76Y/uRP/oS+8pWv0O23305bt26lN7/5zTQyMkIf+tCH6M1vfjO97GUvo5e//OVERHTuuecSEdGXv/xl2r9/P91yyy00Pj5Ojz76KH34wx+mRx99lL797W+T4zj08pe/nHbv3k0f//jH6b3vfe/Jb1tObCwls7OzdOmll1Kz2aS3vvWtNDQ0RB/72MfoJS95CX3yk5+kl73sZSz+3e9+N7muS29/+9tpeXmZ3vOe99DP/dzP0Xe+850fe09mZ2fpkksuIcdx6Jd+6ZdoZGSEvvCFL9DrX/96qlar9La3vY3S6TR97GMfo8suu4x+67d+i/70T/+UiIje8pa30PLyMn30ox8lz/NW3Rc/yqte9So688wz6d3vfjd97nOfo9///d+nwcFB+qu/+it63vOeR3/0R39EO3fupLe//e104YUX0nOf+1z2+Xe9613kOA79+q//Os3NzdH73vc+uvrqq+nBBx+kdDr9pO1+05veRB/96Efplltuobe+9a104MAB+sAHPkAPPPAAffOb36R4PE5zc3N0zTXX0MjICP3Gb/wGlUolOnjwIH3605/+sX1K9MMN7m233UZXX301vfnNb6Zdu3adnD8nzn+CcrlML3jBC+jlL385vfKVr6RPfvKT9Ou//ut0zjnn0HXXXfek1/jCF75Avu/Tz//8z69Yn5WQf0B5nkcDAwNERNTpdOhTn/oU/X//3/9HRD/8ifeWW26hmZkZGh8fX/HcJ/7IOXG+p8rDDz9Mvu/TBRdcwI4nEgk677zz6IEHHjil8xrGfzqRYfwUcN9990VEFH35y1+OoiiKwjCMpqamol/5lV9hcXfddVdERNFdd93Fjh84cCAiougjH/nIyWPnnHNONDU1FdVqtZPH7r777oiIoo0bN8JnR0ZGokqlcvL4O97xjoiIomc84xlRr9c7efw1r3lNlEgkona7HUVRFNVqtahUKkVveMMbWJ1mZmaiYrHIjt90000REUW/+7u/y2Kf+cxnRueff/7J8vz8fERE0Tvf+U7oq2azCcc+/vGPR0QU3XPPPSeP/fEf/3FERNGBAwcgfuPGjdFNN910svy2t70tIqLo61//+sljtVot2rx5c7Rp06YoCIIoiv5f/5955plRp9M5Gfv+978/IqLo4Ycfhmv9KK9//eujiYmJaGFhgR1/9atfHRWLRda2d7zjHZHrutE999wT3XnnnRERRe973/tOqS/e+c53RkQUvfGNbzx5zPf9aGpqKnIcJ3r3u9998ni5XI7S6TTrnxPtXrduXVStVk8e/8QnPhERUfT+97//5LGbbrqJja+vf/3rERFFO3fuZPX84he/yI7/8z//c0RE0fe+9z21756Mubm5KJFIRNdcc83J+xRFUfSBD3wgIqLob//2b08eu+KKKyIiiv7u7/7u5LFOpxONj49Hr3jFK37sdX71V381IqLogQceWFW9PvKRj0B7TtwH+d+P9tcnP/nJiIiiPXv2RFEURdVqNUqlUtF73/tedv4T8/a2226L5ufno5mZmejrX/96dOGFF0ZEFN15551PWrcT40muIz/6bz86fk5w4403RuPj46tqv2E83dhPwMZPBTt37qSxsTG66qqriIhOZv/dcccdJ3/ueyocP36cHn744ZM/25zgiiuuYMLwH+XGG2+kYrF4snxClP7a176WYrEYO97tdunYsWNE9MNvoSqVCr3mNa+hhYWFk/95nkcXX3wx3XXXXXCt//7f/zsrX3755bR///5Vte1Hv2lqt9u0sLBAl1xyCRER3X///as6h+Tzn/88XXTRRfSc5zzn5LFcLkdvfOMb6eDBg/TYY4+x+FtuuYUSiQSrPxH92DZEUUSf+tSn6MUvfjFFUcT66tprr6Xl5WVW/1tvvZXOPvtsuummm+h//I//QVdccQW99a1vZed8qn3xi7/4iyf/3/M8uuCCCyiKInr9619/8nipVKLTTz9dbcsv/MIvsGSEG264gSYmJujzn//8k7b7zjvvpGKxSD/zMz/D2nz++edTLpc7OT5OJOp89rOfpV6v96Tnk3zlK1+hbrdLb3vb21hizxve8AYqFArwM34ul2OauUQiQRdddNGK469arRIRPWkyxlPhU5/6FH35y18++d+P/sS7c+dOuuCCC04mgJ2QUzzZz8DvfOc7aWRkhMbHx+nyyy+nxx9/nG6//Xa64YYbTqluJ2QHmk42lUoxWYJh/CRjPwEbP/EEQUB33HEHXXXVVXTgwIGTxy+++GK6/fbb6atf/epTFl0fOnSIiEjNIt62bZu6OdiwYQMrn9gMrl+/Xj1+Qr+1Z88eIiJ63vOep9blR3VERD98iMifYgcGBkAP9mQsLS3RbbfdRnfccQfNzc2xf1teXl7VOSSHDh1SrXfOPPPMk/++Y8eOk8dlX534ue3HtWF+fp4qlQp9+MMfpg9/+MNqzI+2J5FI0N/+7d/ShRdeSKlUij7ykY/AT7pPtS+0e5xKpSAhoVgsgo6QiGj79u2s7DgObdu27eTPjhp79uyh5eVlGh0dVf/9RL2vuOIKesUrXkG33XYbvfe976Urr7ySfvZnf5b+23/7bz82aefEWD/99NPZ8UQiQVu2bDn57yeYmpqCfhwYGKAf/OAHT3oNov83jmu12o+NWw3Pfe5z1SSQSqVCn//85+mXfumXmKb0sssuo0996lO0e/duOu2009hn3vjGN9KNN95I7Xabvva1r9Gf/dmfndIfjSc48UdFp9OBf2u32z/2p37D+EnCNoDGTzxf+9rXaHp6mu644w6644474N937tx5cgMoH1wn+Pcs+Cc4oStb7fEoiojohwJ/oh/qADWN0o9+e/jjzrdaXvnKV9K9995Lv/Zrv0bnnXce5XI5CsOQXvCCF5ysy380K/WJxom6vfa1r33SjM4TOscTfOlLXyKiHz549+zZQ5s3b2b//lT7Qqv3qbTlqRCGIY2Ojj7pN1gn/hhwHIc++clP0re//W36l3/5F/rSl75Er3vd6+j222+nb3/722CbcqqcanvPOOMMIvqhRu68885bk7pI7rzzTup0OnT77bdDIg7RD9eCE/ZIJ9i+fTtdffXVRET0ohe9iDzPO+n3J3V8q2FiYoKIfugbKJmenqbJycmnfE7DeDqwDaDxE8/OnTtpdHSU/uIv/gL+7dOf/jT98z//M/3lX/4lpdPpk980yexG+S3Hxo0biYjUzNTVZquulq1btxIR0ejo6MkH0b+XJ9volstl+upXv0q33XYb/c7v/M7J4ye+hVzNOTQ2btxIu3btguNPPPHEyX//9zIyMkL5fJ6CIFhVP/3gBz+g3/3d36VbbrmFHnzwQfrFX/xFevjhh9k3sKvti7VCnjuKItq7dy9sXH+UrVu30le+8hW67LLLVvXt0SWXXEKXXHIJvetd76J//Md/pJ/7uZ+jO+64g/18/aOcuDe7du2iLVu2nDze7XbpwIEDazYmr7vuOvI8j/7hH/5hTRJBNHbu3Ek7duxQ7Z/+6q/+iv7xH/8RNoCS3/qt36K//uu/pt/+7d+mL37xi0+5Djt27KBYLEb33XcfvfKVrzx5vNvt0oMPPsiOGcZPMqYBNH6iabVa9OlPf5pe9KIX0Q033AD//dIv/RLVajX6zGc+Q0Q/fNh5nkf33HMPO8+P+n0REU1OTtKOHTvo7/7u75h7/7/927/Rww8/vKZtuPbaa6lQKNAf/MEfqNqt+fn5p3zOTCZDRLjRPfHtjfy25n3vex+cI5vNqufQeOELX0jf/e53mTluo9GgD3/4w7Rp0yY666yznkLtdTzPo1e84hX0qU99ih555BH49x/tp16vRzfffDNNTk7S+9//fvroRz9Ks7Oz9Ku/+qvsfESr64u14u/+7u/YT6Cf/OQnaXp6+sdmz77yla+kIAjo937v9+DffN8/eX/K5TK05cQ3bdrPkSe4+uqrKZFI0J/92Z+xz//v//2/aXl5ma6//vrVNG1F1q9ff9IH78///M/h38MwpNtvv52OHj16Suc/cuQI3XPPPfTKV75SXQtuueUW2rt374qZ5qVSid70pjfRl770JXrwwQefcj2KxSJdffXV9A//8A/sXv/93/891ev1H+sfaBg/Sdg3gMZPNJ/5zGeoVqvRS17yEvXfL7nkEhoZGaGdO3fSq171KioWi3TjjTfSn//5n5PjOLR161b67Gc/C/ovIqI/+IM/oJe+9KV02WWX0S233ELlcpk+8IEP0I4dO9b0lU6FQoE+9KEP0c///M/Ts571LHr1q19NIyMjdPjwYfrc5z5Hl112GX3gAx94SudMp9N01lln0T/90z/RaaedRoODg7Rjxw7asWMHPfe5z6X3vOc91Ov1aN26dfSv//qvTDt5gvPPP5+IfviNyKtf/WqKx+P04he/+OTG8Ef5jd/4Dfr4xz9O1113Hb31rW+lwcFB+tjHPkYHDhygT33qU2v21pB3v/vddNddd9HFF19Mb3jDG+iss86ipaUluv/+++krX/nKSXuQ3//936cHH3yQvvrVr1I+n6dzzz2Xfud3fod++7d/m2644QZ64QtfSIVCYdV9sVYMDg7Sc57zHLrllltodnaW3ve+99G2bdvoDW94w5N+5oorrqA3velN9Id/+If04IMP0jXXXEPxeJz27NlDd955J73//e+nG264gT72sY/RBz/4QXrZy15GW7dupVqtRn/9139NhUKBXvjCFz7p+UdGRugd73gH3XbbbfSCF7yAXvKSl9CuXbvogx/8IF144YUs4ePfy+2330779u2jt771rSf/cBsYGKDDhw/TnXfeSU888QS9+tWvPqVz/+M//iNFUfSka8ELX/hCisVitHPnzhVfFfkrv/Ir9L73vY/e/e53M1nJ7//+7xMRnfTn/Pu//3v6xje+QUREv/3bv30y7l3vehddeumldMUVV9Ab3/hGOnr0KN1+++10zTXX0Ate8IJTap9h/KfztOQeG8YqefGLXxylUqmo0Wg8aczNN98cxePxk9Yh8/Pz0Ste8Yook8lEAwMD0Zve9KbokUceARuYKIqiO+64IzrjjDOiZDIZ7dixI/rMZz4TveIVr4jOOOOMkzEn7CT++I//mH32hPWHtJPQ7C1OxF977bVRsViMUqlUtHXr1ujmm2+O7rvvvpMxN910U5TNZqGNJ+wxfpR77703Ov/886NEIsEsYY4ePRq97GUvi0qlUlQsFqMbb7wxOn78uGob83u/93vRunXrItd1mSWMtIGJoijat29fdMMNN0SlUilKpVLRRRddFH32s59dVZ9oNjxPxuzsbPSWt7wlWr9+fRSPx6Px8fHo+c9/fvThD384iqIo+v73vx/FYrHol3/5l9nnfN+PLrzwwmhycjIql8tPqS9O9O/8/Dw755PdjyuuuCI6++yzod0f//jHo3e84x3R6OholE6no+uvvz46dOgQnPNHbU1O8OEPfzg6//zzo3Q6HeXz+eicc86J/uf//J/R8ePHoyiKovvvvz96zWteE23YsCFKJpPR6Oho9KIXvYiNnx/HBz7wgeiMM86I4vF4NDY2Fr35zW8+2U9P1q6V6qzh+370N3/zN9Hll18eFYvFKB6PRxs3boxuueUWZhHz42xg5H2Ioh/aNm3YsOHHXvvKK6+MRkdHo16v96Tz9gQ333xz5HletHfv3pPHSLGgOfGf5Otf/3p06aWXRqlUKhoZGYne8pa3MAsgw/hJx4miNVIyG8Z/Ec477zwaGRmhL3/5y093VYyfEu6++2666qqr6M477zxlexHDMIz/TEwDaPQtvV6PfN9nx+6++2566KGH6Morr3x6KmUYhmEY/wmYBtDoW44dO0ZXX301vfa1r6XJyUl64okn6C//8i9pfHwcjJgNwzAM478StgE0+paBgQE6//zz6W/+5m9ofn6estksXX/99fTud7+bhoaGnu7qGYZhGMZ/GKYBNAzDMAzD6DNMA2gYhmEYhtFn2AbQMAzDMAyjz7ANoGEYhmEYRp+x6iSQW2+99T+wGoZhGIZhGMa/l9Xu1+wbQMMwDMMwjD7DNoCGYRiGYRh9hm0ADcMwDMMw+gzbABqGYRiGYfQZtgE0DMMwDMPoM2wDaBiGYRiG0WfYBtAwDMMwDKPPWLUP4Gr48l33sXK1WlUu6LNyIY6vIp4sJlh5dCADMYMlfiybSkJMPMbP48YxpusHrFytNyCGHL5PzueyGOL3+Hm7XYhptzusnEwlICZ0eH+0Oy2ISWd4O2Ie7uN9UR8KHIhxHY/XJ4n1yWV5P6dSaTyPy4dRp+dDTCT6kFwPYo728D5L7rn3m6zsYLMom0vxchbvV6fD+2dpcRli1k9tYOV0Kgcxy9UKK+cL2D8UxVnx+uuvh5Arn/ccXr92E2KaLT4WkvE4xOSzeVYOXewgXxxqinFJRBT3+f1ZXl6EmGOzx1l5YbEGMcl8kZXbvRBidu86wMoJZZ6mPf65Q7sfgZixjVtY2UkXWLl59DH4jOQdv/O/Voz5jwR7B9dHV7y+PZRznYg67bY4C46DdpuvUa4yJ/MFPp5iypiD18krc/K/Kq7L17V33Xrbip/53jc+D8dmZuqsnEjgmhVL8PvTbuGzyiP+PBsqFSAmFINsaRnP02zzMdXzcWSGAb/v2m2PxfmzwVUW7ESWz/dCEetcr/H+aTZxfWx1+HgOWvgMTolnVaDUpx3ytscSuE1KiWe31nZXHA0j7MMgFHNZ3hwiaou5/ItvfINytVPDvgE0DMMwDMPoM2wDaBiGYRiG0WfYBtAwDMMwDKPPWFMN4K7de1m5ulSBmFyc/8Y9VcQqFDtc/xDEihDTc7nWqlrHX+Edsb/1Q9zvtoUWrNVG3UAoNC6LMTxP0uUxvtAWEqHmLqFo7jpdrsfqBVgfP8/7J57CPgwDfn0vUvpHHKv7qN0rC11QUtFaOq7QBTmoJYqE1qLdxf6Jbb0IjkFMTOgNO9g/gWhGGKJGKp/nOkFNr7pcLfPzELYrnuTjsNXGPlw3uY6VY3HUEu7ZdZCVu702xOQLvM6JgQGI8Xu8X7/1wMMQ89Cefay85YzTISYe8rGZxNtOQ6NjrLx5fAqDhA6318N7kRA6oeMHpiGmPD8jPoMVCoVOKeqgtnElpKbr6cZRNICOUAp2mtjOJx7i972+XIGYXJ6Pw5GJCYjJCL1xXFmznH4S/a0BgwXUuPlCmBtPpCCm2xVrS4R6zMHiICuPDOG15HlSOdQt18SYKi/h+iiHZjqNc7LR4PpCqb0mIgqEzD07gu0ice6YNi8ivvbVWrgWu+LZnVP0/BnRr50ePqtC8ayMFO1eJPR9jrK2yPXQVfT8cQ+fO2vFT9ZqZxiGYRiGYfyHYxtAwzAMwzCMPsM2gIZhGIZhGH2GbQANwzAMwzD6jDVNAokJEWbaRRHmSIZfcryEAtShAS5O1ox8PSEcl+J3IqKeELv2AhSOhkLA7Ma1hAop8MRrpXNcpOr38FpxIVyPQiUxQ4hA3S4KQEP5uUgRjop2xNyV26WZYkqPV2mc/cNr8esnEijibba4qLirJG+sZjAWCnxsLC2hgbMvkgEqigB+NDHCyoODKJbuiIQOX2aXEFFaGC97LgqhYwk+xp94Yh/EFPLnsHK+gKbYQ0Nc4B3TEhZcLtJ/+NG9EPJP//eLrPwzP4vncULejkajDDHj4yVWHiiWICYjjISzSRS3T43xZJKpkVGIOX6Q359HHkPRtefxESSNs1cDmBr/JwNXV+vD295TTHErs7OsfGwfjgOZYLL+9NMgJpvn9y+ZwfU6EslkjubO/l+UU2lrpJihF4QZclx7SYCYk34O14iCTBBMKkbHIsEsmcT5n87y+ZZJ4+rcFkbLxWIeYpoNPl5qNcW8OuLnLiiJhiSSJZJKv+dyvM/ScSV5IuJr+MgoJtJRjH9ufm4JQuo1MeeU5A0vwY95MSWJ0OPPs3q9DjGRi+NlrbBvAA3DMAzDMPoM2wAahmEYhmH0GbYBNAzDMAzD6DPWVAOYFL9VpzO4v9w4xn/zniigXiwnDgUd1Lg0u/zcvS5qZXrCDDmexN/gU2muSdJemi71a8k4tquY4+dpNtCcNRAvbe8pZsjy5dDJBGpBIvFSaRc0imiYrPrbii5zFTmLNMF2HUWTFPB2+R2sjyOMMj3tPKsgHuf9oelDm02u/UoksGGeaKyj6DFDqdVR9I9+xMfU6ChqCd2Ij9+BYTSC3rR1ktdPMRt3hSl3T6nPzHFumFxerECM3+IaoKXZGYgZHOLm1YtzqLXsNvl5FhIYE4L2FPt5anKclQcUXXBc3K9eZghiPDHIY16NlVHBiTzt+jUx/zUJoCPmf20JNUq9Bm/7YB71YlKT1Gmg/mi5vMjKxZFh5Tx8Tj7tffgTTjKGa7oX432WUDR34h0B1GmjxrVS4VrdTHbla2k+3oFY0zNpfE6nEryOySTWOZvhL3EoFnBuOz0+DqXRORHR6BA/T095aUEv4nUeHkRNYrfHdYvJFGqSpaY15uBaXMvyNb3Tw/okQMuIkzmf5fNyaRH3KB3l5RRrhX0DaBiGYRiG0WfYBtAwDMMwDKPPsA2gYRiGYRhGn2EbQMMwDMMwjD5jTZNACiI5IhlD4ehwnosuhwsoUo17XIQZKcbLkRBUukoGg9zdOoovZExcK1ASKkKfizADRTXbqHNBfLfTg5imMM5s9zAmkRDCUQ9vkQsiWSXJQSSBhMpeXxrBppTEg64wDQ2UxJWuSLYJFbFro8XbWm/jeTaeCYfwWt2VpfzZLBcaJ+N4rbZIYAgD7MOeuIfNqA0xqSSPiXoY44jxOziIQuiMmBdaglB1mQuPjxw5BjFHD/FjgWK4vWEdT/CYPj4HMcem51l5aWYeYrLCFLiQxeSWeIzH+EqyzXKZC9eTGUW4LsZUz8c+XDfCE0NKGW7yWlvG/no6UU2n4Zgylyo82ebArj0QM3/kKCujmT2RL9aEKIlJILlxbig9umEDxuT5Oq+1yxJD/h+LS2iqHsvxe9FoY+Kj5/FnQxgqBsGi76s1vKfJFL9fCcUwWSYs+j7eUzBeb+K15AsJQh/rHIkEzkhpViZdYmVXScyQ2wQvgc/OmEgIbLdxvY45vH8Gi5jYl0nx5I2Wj0mfaZHg0VbW4qDNP5fJYrJWVjm2Vtg3gIZhGIZhGH2GbQANwzAMwzD6DNsAGoZhGIZh9BlrqgEcFFqQjKf8np3hMUnFnDkpDEpDRSojZQKui0Fg5KucKBT6taiHv+WHAf/t3gdzW6Jqk7/kOvAVnaDQAPQC1AD6IW/7cgvr4wpdUCalGEG3uEmoZpSdJK4pKeXRODOZ5DEdQm1Ku8Pb3mxjnSs1YT5cRz3ERjiCtIX5aTyOOtO0MPfutNCguNXkdSyVBiHGG+Dn9hwcz0lxrZLyEvetU1yLNjpchJhQjM3Z2UWI2bf3MI+ZmYWY2ePHWXlxDvV9W7dtZ+XjHdRVzs0cYOVqHXVLc7P8Wp7y92Q+xbV6GcV4dWGGz6dUFvsnneDn6YVoAF5Kcw3ihgluMF1Dv+unF0UXJ7XMnSbOt31P7GblQ3v2QkxF6CoXKw2ImRPHUkWc/7lx3odbzsC5lM1J7ad9r/DjaHVx3W/P8fusacEGB/iYz6Zx/PhC69lTNLdSz4eaciJXjE3tBQCBeHYGijlzJHSKskyE60ZM0e5V63zcaZpSeaitPMsb4vnhKHrVTJyvNa6iD5fNkHp6IqKW0HH2FP16T5jya87v8SQ+U9YKm6mGYRiGYRh9hm0ADcMwDMMw+gzbABqGYRiGYfQZtgE0DMMwDMPoM9Y0CWR0gIsn84pAP58VwnrFnNmLCQGqYgzZ6wlBZYDGuUmX72+9hGI6LbbAnUg5j2hHRzFVLS/WWLntKw3r8XPntN73eHJEuYnJEtJDWRpnExElhXB0KD8AMdkEF30HrSTExJLifqUUU2VhpqncCoqEWbWWDLAaWkLEm0jgGCMhWF5aqkGIJ+57LofC7Jgw0y4W0Hx463aeujKeVhJ7qlzAnFaMV0PRZ0tLKLbviiSiI4enIebg/n2svDhXgZhnbT2blceGcWxMiASKpQnMoDh+mBsrV8tY5/ES77PRQRS3H5vh5y7Poul0WXRrmMB7USjxsTAxegrmw5oLrQxRTGhBI6959IrqhIqwnsT6M3f4IITsfeAhVl6cxmSg+UV+L/YfXYCYRZGYNSorSEQtYYbuB4rQXwjXVc9naCq2XZr7KyFwckcxwdcOrYx2sdXEPPXvUDJKsl23zO+7j8s++cIcXlv7PLHuhz3sjG6XJ0cEPi7YGZHMlk3js1Ou850OmirL57Sr7Al88UIEN4F96otnp+8r66xYH0NlIMo5F1PmsjymPMkpcPgE1xJg2mLuaIkr8v0VjvJCCy3BZK2wbwANwzAMwzD6DNsAGoZhGIZh9Bm2ATQMwzAMw+gz1lQDOCL0PinFNDgtX0QdW1lPEyi6E2mYmlLMIyOHH/NiGON4/Ld8P6ZoFNr8+gtLLYiZX+b6h1YPlQMDoq0XnDYBMSMDXBv3g+Ooq9o9x1+6XVOMfOWLwmMF1BbI3gja2C5H6g1RJkiJBG9rXLmnaaElTKVO7W+PjtAbtpN4vxJJ3rKYhzFSV1GroU5wZISbQ0+tR7Po7ZtGWLm861GI2fsoP7bt/OdBTCj0V6Ui6vI6LX4zDh1F7deeg9ycOS7N0IkoEi9k92Koy5sYnWLlwfw4xCQ83h/1gSrEbB7lL1KfHEPt3tQGrvnbswf1hseO8LYGiiap2qyw8qMPfYeVN42V4DMSxYcVCBVNjisMd51A0biJhS1SVt/aIjfu3vPA/RCzuJ+bdC8vViBm3ww3Ej+2VIeYrtCUjSgG9yNjfHyXSmjSLVmNBA/0fkQURb4oK+eWnaZeTAqrVq7P6nRWa6PFSqRxDiSbXC8mX1BARNTt8GOeqzw7hX4tUnSdcbGIJ5X1OiZ05QlPGaxxcUy7X8Tb5cXwuZjJ8/Wn20UBpOfyz6VyqCFvVfh+w1ESDOT7GTxFl1fMcGNzR+nDbp3PJ+08qTTv50h5EYV8NvUUk3BXuc9rhX0DaBiGYRiG0WfYBtAwDMMwDKPPsA2gYRiGYRhGn2EbQMMwDMMwjD5jTZNABgpcPBkpJsYxIcx0XBRqdoXpY6CIMKV40lMMFKUOWxNvy/oESpfMLTZYeXYR29UQhpudEOsz5vHr7xhDQ9CpId6HFKHYtd7hbZ9eRvG9IwxtK8sViGlHXDSbiiuC4TwXLGdTipm2SMDJ5pQYkZjRVcxHV0Myxc8TV0yV5bGRkVGIqZTLrCwTWYiITj9jMyufdeZ6iMkmeDuOzM1BjF/liSuuVCITgUvw4CAmgZTLXHhcbaDx6nKHj7FxzXBbJIF0WziefV/MFQfnhRfjIudMBsez6/Dx3Gnj35xT63k/p3OYcFIo8MSQqjLfw4jP070PfoOVN41dBJ+R+KsQ+ntKVoFM8Ahd7Tz8WLeBSVd773uYlQ/84AmIWSzz+X5krgwxM3M8ptbAe1waKLHyWefsgJjtp5/Oysl0DmLk9wiqf7M4GimG24FIfAiURJq4cHCXSQ9KdXSzaKjgalJXTslhGtCM+4sFnghRq2BSWjHHY+LKmtVo8ySCXhsTBD3xHHKVJDk3Es/FjrJeh/zcoZLA4AhHdF9L8BDJmYoPNDWbfK6MDGAy0kiWj02/h9eqC7No38dxmBKJqqk4Ps98MX67dVyLfTF+5WeIiBpNbsrdaeF5NHPotcK+ATQMwzAMw+gzbANoGIZhGIbRZ9gG0DAMwzAMo89YUw3gkNAt9fCnc3LF7/1dRTfQI/47eORor2PmqK9wF/oQeW0ioq7Q6i1WUXuxWOXaIk2+lhIv5k5Lk0wiKiT5BztNvNaceFG3F6DmZlS8TLwXYOtbHa7va7VQb1Trck1ZNo11Tre5jiHewxhX6GfSadSUJJM8Rqvzahga4ObDSUWTmBK6t3gG+9ATprdbtqBOcIfQAOZzis60JXSUSYwZGuR6lWQcjWBJGHdHhNodKYV1k9guJ8WNlzMl1OVJ0+nI70CMH/JjvQC1KbEYHxsO3nbQei5VFI1Lhs/TTB71PYPjvM6holt0Iq5JPNTEPlyJmfI0HEsm+BgrpbFP5ZU0LWFC/L19bNc+iNlz70OsvHhsEWKOlLk5/MG5CsTUa8KA18Wbc+4zuObvqudfATHrprjuNVS+M4iENk5TLEn9ta8YHVdEu2am0RC8VOLPmHXrJiHGI/68iBRHadAFrkYDqIsbnzKBok3LZ/ia5bcaEOOKp1wkdbpEFBNt90K8Vq8rnq+a9jshHP97eL/iwqDYlZ8hIkcI+kIHn/dByOuTUM7jC+1eVYwVIqKp4SFWbit1jgndYqTc906PPyvdSNE2hvzcXUVrWRN6x26I51le5s+PbgfX4mw2C8fWCvsG0DAMwzAMo8+wDaBhGIZhGEafYRtAwzAMwzCMPsM2gIZhGIZhGH3GmiaBZPJcgE6KUaVMGGg2mxDji+SNQMm6cKUiVzFLdIWBo6M0d3mJJ2IsLqGpcjLGr59QkhzSaX6twQEUiheFVPxYHYWjoRCB+mlslzSqHB8sQIzrcSF918d+XqxwIW0QKIkHkRAR+yjil0Jaz8W/K2IxT5RRwLwaa+hEjPdzr4PC2kyGG6aWBksQs33bBlbeMoV9mCB+7tYyCrNjDhcsa/lKnrhfbkwJEqL40EExMAmxdjyFSSCxBD8WVxIW3Bi/P+mUIpIXQ7wX4t3xPCkCx3nhiTVAi6l2+XmSyqq0LJKjajUUgQ8IA/LCwDCeaAW+89A34Vg+x/twvDABMaHHK50qKIbJZT4HH/oaXmt231FWXqrUIWb/HE8Mma/juKSI9+lZZ2yBkGuuu4qVzzz7dIhxRTJbEOJYgbVYmcmhSHKansZkm29889usvG/vAYjZvGkjK//M1Zi4MjbKE7q0JJBQJP85Hs5JR6xrWg6IfJ6thppiAF7Mc6H/4DAawfu+MF5W2iW/0nGV52JC3NNISchpNfiYSsaUxL4MT2bzIuV5nxLG3TFM8Kgu83Y5SjZZLsufZ0uLOP+7Ob5GFAuYTBaE/PleVsz02y1+HieuJJOJ+aV4e1MgkkD8AJ9VMWEY7zt4T/NZ7LO1wr4BNAzDMAzD6DNsA2gYhmEYhtFn2AbQMAzDMAyjz1hTDWAgDGalhoKIKJKWzcpL06VezPOUaopr9ZQXLUfiJddRiNoCX7w8O0f4O/3kEP+cH6DWoTDITSgnx1BTJk2DYwk0BG7UuOYnmcI+9Fx+/ZHsIMSMjo+xsh+itnHvPm5EOzODprMxR+hDZJmIQqH9kC8SJyJypcZG0ZStRgNYr/E+dJXxU8px/dX2TUMQMzbM9SFRB3VUvtCnZnLYLr8nXgquvOw8lhRa1Bj+3RUIjVQYKb0h9CGJONYnKeZKIoFzJ5fnmpLiEI5DqWHrKlqZeIqfp6WYuleFpjXupiDG7QizcWWMLYuXwbdbqGkloQEsDaO590ocOobmzIk4b+fB6BjEZNK8v0pZ1AAee2gvL3//cYgJlni7Di4sQcysMKuv+TjmNm/gBsnX/eyVEHPRZeexcraA96YirtVW7nGxyNuaSOK4bIr7tXv/foj55je/w8rHZ+cgZnpxgZW7EeqzzttxFisPlVBPJ42giwMliCmWxDHlGXMqj9Cecppag/fPiKJblkSKKrHd4f3herjWOGKlDRUj+F6bjylNthx6fLxEij601eHrRtLF/up0hQ5O0fynUxkRgy9RWF7mz86c0IIT4QsASHlWBUIT6Si5DDGx9joO6jrTQsw8mMM1QV6r3cJ1NplU3qixRtg3gIZhGIZhGH2GbQANwzAMwzD6DNsAGoZhGIZh9Bm2ATQMwzAMw+gz1jQJpNMVho49FCeTEO12uih6DIWYtKckXTSbXLhaV0ThSWHYnE6goWLC4Z/bPI5dMjXGRfJtH5NJRjZu5tf2UBBbqQgD3iIaVWbrXCg6mEcj344wxc0NYRJIcYgbi7bbmExSq/HP1RXTWfkp10OhOBHvj1BLyBGmpaEi9F0NzToX227ZNA4x552znZXHhrHOzSoX13uKqWouw8dLMoF/L1VafIzH4zg2UmkuRnbieJ4QBN0Y44okkGIWr7Vlkic+nLYFEyG2bOIGyRnFdNZN8fGT81HkvBDwY+UO3ve2WBPCHt73eIL3jxvgeeIOP4/fwXUjEEa5EYzelQl7KLhuizyDBuE8kcL1qjB0JiI6ct+jrNxZwMSs+SofT0cqGNPq8QrFUOtO608bYeXsEI6VfYf3sHJ8GufJski6qlSx7YUSX7NyOVxnK6IdjzyyC2JqLT6eEllMTqqHfO371oP3Q8wTe3az8nARx/fWjVOsfNHFF0BMriCur/kuu099HYsnsZ/bHX7fa03s56QwUYfEOiKKRIKHEkKecKt3PCXJQKxRWkJnUzyHuj3sID/kSUNdH9c13+fzPXKUdUS8kKDbxeSfijBaTymZK9KIWskhomaL17nXwSQZT3RsIob9UyzwcZcvYRJISyQadjvK2FASC9cK+wbQMAzDMAyjz7ANoGEYhmEYRp9hG0DDMAzDMIw+Y001gI44W6hoAqS+Lx7DKkjz2qVlNB+dX+LGvY0WaiayGX6etqItyIVCU7YdDZzPPofrzKYXUX+UneC6qpERfGH8ojAxzWex7dJE2VP26LUafxF25KEOpROWWXm5gToGP+TXyiuaRN8Xeogeah1CaTaq6EUcYfzsuYoZMhxBJtZxU+crr7gQYoaKXEfRqpUhRrwPnfI5FFL1OtIMVblfcjzLExNRLC0MU5U/uwKh7/Nc1GwNCMPdq577LIgpz/N5MTmOGtKzz9nKyrkS3vdQ6MyOp/Hu7A8rrNzOlSAmlebXzyovaJea0WoT9bxPBHzMe23UmRXTvO87ioHrStx712NwLCbuaS+Bc6kU8jFfWlJ0OwvcvHZZ0dMdWOZtLyuaSeGBT5s34VoztqHEyk8ceARivvWde1m5WsN1LRJ6MS+GGqWEMLz1FOml1LgtLio6StHULWdsgZj0MNem1itoCBx2+Jp1fG4GYgYG+Lg8euwIxNRqFVaOJ5QXCfiKiGwF0kkcu66YA50Oam6jUJg8u4oxvehE18EYT4znhIftSgvdcjaP62MgNLatFvZFp83b0ethu6TEv9fD8yzX+Oe6ii4vJ4Zmt43j2RcvemjW8VrVOp+DvQDPMzjA9wnr1qEWPZPmGtJEGu97R+jKm8raFwT4ubXCvgE0DMMwDMPoM2wDaBiGYRiG0WfYBtAwDMMwDKPPsA2gYRiGYRhGn7GmSSDSADRwFANeYeAcdTExoyVi5mdRxC+NRSPF0TESJtOR4uQ5PsHFnMMTUxCTKgkj0cYCxMSE4HPs9LPxPEv8c0l/CWKCiItd221MgBnIcQGz72O7usLY2MvhrU4VuXB1dBFNZ8vz86y8tIii645Qpfd6eE9dYYydVEy5V2N3ef4FZ7HywACeJ+zyOkYBCum70mw4hn8LOULk7MpkFyIKetJ8GO+FJwxJI0Ul74skGU3gnc/xMXbxhWdBzBFhQDw3fxzPk+XGrwPZLMQEYu4kp4YgZnSQC8OjENuejPE6Z3KYZBUQvxfzi4sQM1TgdZ5WRNfjE2OsvH+If2Z+8RB8RnJ87zE4NjnFzbT9TgVi6kKkHusof1uLeXF8eRlCykK43iScSxs28nZeeCmOg02n8f7R1sd0lt+vXBWvVa3ytbhRw+SNyjxve30Z15FaXZzHxzkwLBLnBotocJ8f5GM1lC7dREQuf+40FBP8HTvOZeWpUTSL/sqXv8bKTeWlBek8mlWvREpJTnJDvj6nkkrCmWyGkmwXE2uUo5hXx4RpcTKB1xob4/N989b1eB5hFt9r4To7P8efH4cO74eYkUHeh50Onqde5cltg0U0VR4b5PPdi/A8x6YrrKwlisgkFGlmT0TkiYTAwQFMtpMJgo7ybIhJs/MQ6ywTZ9cS+wbQMAzDMAyjz7ANoGEYhmEYRp9hG0DDMAzDMIw+Y001gPXlCj95rwExnnQx9ZTfxT2pKcPf6eV7nvMZ1DHFhM4rrbyweWzjRlZOjJ0OMbuOcQ3i/HGs89h6rgGoVlG7N7ThNFZ2XVS9BV1+rUKIMS1hftproilmPMW1cX4c+6fn8GORYtw9vY+/tP2he78LMQ1hcOsqZsiR0NO1T1HWkE6Ke9hDTVJKjJ8Hvr8HYjZuWcfKI6P4t5ArxksQohakJzQkml4jEFqdMFJemi50io6Peiz55nI/VAxKhZ4mnZ+EmK54iXvQw7Hqi3maLqBGKlviOiGlxiS91wOlXUHA2zU0MgwxI+JYUzFRTqWEeW3ETVW/9vWVNYBOgHNpw8QIK3eWcB2pVfi96HVRy1Ou8fVwQdEfdSPZF2icffFl57Hy6adtgpgkl0NRq4VrcUaIygojqM/s5PhYXZjFseIM8HuTjONcml/ia8Te46h/7goD5/JRNHAuz/Hxs6xoEoV3PVWWULe8MF9h5S1TaKZdKXON5sEjOH4K4v6kaWVNYD6NxsuRMHAuFXG9JrFuOIqZvlhmKZDu2kTkiDc2FBUT/Mlxfk83rMd1JJniOrzGMpo8Oz3+/KovYbvGhHZXkcFRs8HnSqeDa/FwkbfL7+KzsyGelfU2Xqwd8H7uKRVyxDMm9HHdiMT+I+zi/eoJXWCrhWtCu83PrYyMU8a+ATQMwzAMw+gzbANoGIZhGIbRZ9gG0DAMwzAMo8+wDaBhGIZhGEafsaZJINLfNmg3IUaaxbqkGB8KAXpdmvYS0VJTCDUV8+GRND/P+u1oZnnmBRexsqckSxz6Lk+EmCihwTUJkfXMocNYn03bWTk9iEanjsP7rF2tQEzk8GNOAgXeDSEm9QgF3gNDG1g5nkKD0mabJ6U4OUXEv8xFqo7iPtoTymxHESejNBrJuPzcxRQaQU/vnmblaB7FydlJ3taUYtzb83iM4yrqZNEuzQg6iIlpFmASAQkz7yDEfpbG1Nrfb1GS92Iuk8AYh38ulJkaROSLdkhTUyIiX6rtFWNaaZ4diysxon8iJUlGmuAmPcV4NeL3K51FcftKJBSBfrPD5+TcIhrTN6p8jMUVV/OFZX6eupI142Z4nZ9z+XMg5uLzn8nK3WYFYo4cOCZiUFzuiXlar6Mx9dI8N3VeXsY5EBNmyDuehYl0V1xxKSsP7T4CMV/4P9x4eY+SVCCTYrw0rsWOy8dTMon39Mgxbph+1un4bEik+drSVJJ2vGWZLHVqSSDSJDyXwHkbE+bDWhJIN+RzsuspBuBJPsaGB0oQUyzwBI+EYpTvd4S59zIm24QtHjMxiMlk46N8LmtJIK02vxc9JQlEeipThPN/doFfv97GSVgTiSFJwjGWSvH701X2OjGxXvvKur9Q559rtTGZRCaGYDrOqWPfABqGYRiGYfQZtgE0DMMwDMPoM2wDaBiGYRiG0WesqQZQSr/CnvKibqkt0ragXfG7vGJQnBa6jsEcNmXTFNe9nXUhalNGtnAd3PTeoxCTdfnv8uOjoxDjCPncsKJ1CMVLpds17B9f6J96LeygIORih1lhwE1EtG/vXlbO5lDHsCPDdRW5GOoEez43Wk0VUSc4KF6+HjpY51AY4/qK+TAeQQrStFQxZ241ua5iahKNhXM5Pn7CAEVbjsfrHCrXktqPSInJ5Xg/u4QxrsvHvOsoL4N3eZ07XeyxapXrcOpV1OU0l/k9jXt4rZTQomXSqKeJC+1eLI7appjQDiUSOH4Sca6nyYCYhyiR4Od25IQjolSCj8NcAU2UV6I4hHNgfnGRlRcqiua2IcaPoi3qiOU2VLRphQFe57lZNEz+l0//Kz9PC82QY0Jb3W4pBs5Cnwmu3URUEzq8RhMFWq5Ye7uEOqa5Ga6bXKpiDAmN3dJsBUIqwhzaSaFWriiMu33ZTiK65657eEwL9Y+Hj3Adt9SCEhElO4qedwWSUixPRK7D25GOY7vkPAmjlTXJPUVjL7W7PeU53Wrx8Tw9PQ8xdXEPHeVx70Z8TI0OlyCmKNbHIMDnfTYt5o7S9ESMHwyUoLFRfv2ZOVwfu2IcOlLDTURJsR6Fmlm0eA4GPrZL9nNT0QBq5vlrhX0DaBiGYRiG0WfYBtAwDMMwDKPPsA2gYRiGYRhGn2EbQMMwDMMwjD5jTZNAQmHY3JbJHEQUE0JtVxGgS0H8SAEFzHmfK05Hh3IQs+3s03j5vGdCjJfhouv68v0QM7GOJ3SMbT8NYhLDU7ycQ5PndoeLS7utWYhZWuBi2+UFjOkJ02nyUIDqeVxc2m6jyHlujieKONE6iPGlILaDZqhul5vFBpGSUCEE5mnFdHo1SSDFEr/Pnov1GVvPReDxCP/OKQ1zsX/g4dVDRySB+FryBj93IY/JEtkUFwxHSsLJokg0OHYERdezMzwh4OiRGYhZrvEYv4PXOnaYmwTnM1jnzZu3sHJmQEkQEmbVtQqOsWq5wsqeIsgfGeZzJaGYe0uT6UweEzxKJZ7sEzaxPisxOIwJQ4FY1xYXMBmgGchEDBT6hyJJJlT6otXmY+yRh3dBTCbO505G+TPeEaL0KKYYC4tkm3QGE8W6Cb4+e4qR9+hEiZXHhtGkO5vi8zYWw/V6scTHwdLSNMQ4Yi5riQfl43Os3A5wbsdTvF27c9g/vsjRyQ9gnYtFbOtKJDXD9DivTyKmJEvFhLFwD9f9lIyJ8DzVZf78aCjJiNU6T0ZwlcSVpkhgmBLJN0REY3ner6VhJZlMJENFypqVFmuUNMUmIoqReOa1cJ6ODPH7lVNMuWPSzF95sYE0UXddrE8k+sxVcjmyKTHnImXdSDz1RKPVYt8AGoZhGIZh9Bm2ATQMwzAMw+gzbANoGIZhGIbRZ6ypBjAmjA9rTfwtP2jz384T8jdwwl1pPo+/0wfLXNeRH8ZXJG845zJWHlq/A2LKC1wvErRRL1ISuqDBrWdATMvjGqkDBw5AjH+IvwC9UUdD2cVZruvym2hUGXN5Hw6MoD5rZILrFpN51CTmMjwmpr30WurnaljnjtDcSM0UEVEg6hwpGhfajIckyQSvj/YXTElokKIINRyUFFoZxWwzlB9TdB6lAT424gHqckKSuiWsz/59h1j5S1+4G2IWl7jWMpVEw+R167hJ+dTGDRAzPsy1OqBDIaJ0gp/7gd27ISY7wnV4S0LHSER04PF9rBxTnN+bmzeysjSzJiKanV9g5dPPOx9i4okKK9dneZ/uOAvnAKDc41qF93uj0YKYUGivNG2zQ3zMOzDASPrkU6eN+qx2nV8/yKBmMpvm9y+M4Roql+dKV2mXqKM0/yYiipf5mpBWdEzNRX6x+WW8x3WhO4sp2lRfmh8rC0Ba6B2HslmIcYS+t7eMZtqDWydYOT6I8y0ltHu0Gtmpsh5J0+BOF5+dJEydpc6TiCge5+dJOzg2Fnu8nxcWsdKLS0JbHcN7GggdnKcYwU8OjLFyroj1kTrqSDG4Tmf4PXUUHWU8FPMrwLEqTacHS6jrXC+0jJGL10pDWzHGEWtAOodtd1w+pjIZxZS7sxp1/Klh3wAahmEYhmH0GbYBNAzDMAzD6DNsA2gYhmEYhtFn2AbQMAzDMAyjz1jTJBC/xwWMis8pOUL77xEKPp1ImgbjeYYnN7Hyjksvhpgt55zDyrEEinjby1zgHYUdiGl1uNi1Vkax+0yVGzZ/9+57ICYpjCFbbUyoSIvkhEwKhbWVBje4bPeqEJMqcuHz6NR2iCkOrGflSEmA6bR421stxTS0xgXLUYiC2G6Pn6dNmHQxsookkHicn0eOFSIi1xXGonLQERHF5ODEqZAQ4uS5mTLE7L6fJzlMKobJfkKYzA4oiT0eF7zvEGOXiGhQmFePjpQgpigMknMZHPNC/07ZJIqTOyJDYLqJbc9O8KSKZBL7cL9IAiHC8ew6vH/SKRRmdzrcGPuMc58FMaksr09zhifoRG1MzJLMz6EgvlLm80tLGIoLw2Q5BomIhG82OUqyREokWXSbmJjR6QnD/UARxHtiDiiGwJ6MUZK3EsJ0emgQx3dY5+vR3ocO4rVEskRb/eqBH0wqiStuQRgvd3G9jot2TBawzq5INOhp91Q+wJRkgHAVSWmS5SbW2fXE/Ylwnc1l+YNQezZ4Yu3tKWMjjPi96PlYn7ZIFAlkdhIRdUK+RqRiGJM8Y5zXTzNV9vixJHpyU0Ks+0GA/SOTDyPlbsiEvJhcm4loYIgnRyrTlBxR5yBUEjXEmHJcJflH3ndlHAZKrthaYd8AGoZhGIZh9Bm2ATQMwzAMw+gzbANoGIZhGIbRZ6ypBjDwuSbAU3QDUoni+PjbuSM+F1de2LzltE2svO30rVgh8dv99OFjELIww1843u60IaZX5yahMwf3Qsxig9e5OoPXKua5HkvTP+aFxqXbQ51gq8mPdRVj0cW5Cj9vEbVNrSbXP/gd1Bst1/n9aRG+/Hw54NqUhKbdEeasCal9WCWhMPvsdrDtQY/fw6YS0xN61XoV+7ktNJH7dh+EmP3f5xq3LeOjEOOIl50v9sYhZupZF7Dy2OgQxAQhb5c0fSUiqlb4fZ47Pg0x7SY/Tz6Dmruk1KJ18MXq2ZBrq1JxFMskRdudUDEkFsbKSUUEJDVSrQ7qlmIprp9Zv54b+R7es7IGsNfDsZJKcz2to4hy6k2+RvQUs+9uj4+nuIPLb6cr1kNFSB0XouhI0XBSgh9zFSFTQsxJUnRMUgu6HKHeOBSatlYd7w25/NyBol+LSXNdZXxnhTm076MWbGaWa7TDBtZnsxgbgyNoEt7t8vHULKN2N9nhfTigrI+SuSrOJXKEXkx5NnSF3tl30Cib2lIrh+tsp8PP0+mi7qzR4n1WV54NnYDHFJOKhlTo3rstbHtCjF9fzgEiclK8XaGP/VMX/drtYsximc/TsrLu11u8rVK/SoRzJ3KVfhZjs9PFtjtC/9xqowG4vBdriX0DaBiGYRiG0WfYBtAwDMMwDKPPsA2gYRiGYRhGn2EbQMMwDMMwjD5jTZNAIiFcjQIUc7pC+BwpCQy+OE8mkYWYRo0LKvc9vhti0gVuXquJS+uVCis7jrInDnkdu00UjnpC3F7Konu1E3CBZ0Ixzg2FULzdVkxDRZeFHWzX/BGehNKtobg0Hud19BURuDQJ7TgoJo9KXIwcovcwJYVRbuwUh95ijQuG7//efogpz3EReLWBQvFeWySKKELxuhBUL5crEBMIk+DaMl4rLZIajizfBzHjZ+9g5W/e8w2I2bVnFytv2LAJYrJpbgQdKvc0IcT2k5NTEDMiRPGtOo6fI996iJW7LUygksk1UYgx3Y5MbsGx0RLi8a9/7WsQMzbMhf1XPwfNtFdCJqAREUXCwdlR6hcX5tVRD/vLEYk1mhE0CTPbdBKTClyRGOIpCVW+mKdRiO2SyUAdJbFGJq7U23j/HFHnntosPi9iPq6zCSGk95W1rzfP1/RuC5MTmiKpIVSMhdNStO9hIk2cxDFlnSWZyLOKZa2qnkcmgSiGwMJwu9nBMeaKzMe4i4txU5j51+t4T8vLPOFlsY7JPz3i1982jslkrni+1yp1iFmsLLByoJghj47yPUFTue9zM0v8PMre4ugsj1koY7tawmg9UhJOYiJ5NZHCOZgQ60RDqXNPzK+ekmA2t8jrfDpEnDr2DaBhGIZhGEafYRtAwzAMwzCMPsM2gIZhGIZhGH3G2moAxc/XyruzKSYFbMrv9K58YbOL1ZQGt/Oz+LL64VGuCSoVixATid/yi6UBiEkS17h5HopK5EvbU2mM6QgdVai80LotZAKOskdPiRdY+8oLv7t1ruEoNxXzUaEX6UWK+agjzCxjqMvJTAgNUAq1Dl35EvAuaiZWMxjnKzzqa/+2B2KOHznCyoGi4fCEriuVQlPVeH6YldO5dRDjJrjxc13Rr9SEzqPYxTF//DjXbN7zddQAHpuZZeXZObynE+Ncz9dqogH4ls2bWPnMM4chJj/Mzao3b4cQOv71e1m5XMHxnMxwDVvQw5hImMV2FHP4dJprmcqzcxBTEprWVOypm437yliJQqFtVnTCjtDlxR2c/57Q4YUhjhWpp3OVa7muK8o4/2UMxTAmjPP6uAlFxyTqo2m2HbGOpANFvxYEP7ZMhH0fKibPYZ0f084Tifve9bDti0Lv6FZRmzYgPqdptmMOGlqvREe57zAWQmxXL+BavZqiE0x5vD4Z5Z5WxXOoqrS9Kl5+0FPqUxjg2vyM8sKGZoOvUZUFfE5/5WtcS6w8hmh8gq9HXUVjW63xZ97QKJp7Vxo1UcbzhEL7GUbKPBWG8akI57ucBl1lXiws8vXZj3C+LyxV4NhaYd8AGoZhGIZh9Bm2ATQMwzAMw+gzbANoGIZhGIbRZ9gG0DAMwzAMo89Y0yQQirjgNB5DUajnciFkXEnwiCW5mNtTzIfbXZ5oEAQoZG17XE3a8lE0H0vwaxUH0Hi1VOAi+U4PEyGSoTC4VtoVCkG34vlIvjAJTSTRoDQmjF8VjTOIdpsyu4SIGk0uqG4rRtl+gguPU2MoZE0O8WO+g8LaujCdjUcoTkbrbOTx3TOsfGRGaVdHJMm00OyzlOFjM5VGE9O4SGBYv/ksiAlD3vntLra9IUyMkwVMOPn+g4+y8owi/HWSPBHC93B+tYSY/PCxaYgp5gqs7ClJRFGHD86R4VGI2fGM81i53cB74QqhujQWJ8JEBy05wg/leSCEhvI8ySsmTXpXQSKBn3FcPgc9LZGFlEkogEQIJeHEEdMirpg8ywQPzQjaFcdkogaRln+nLEiraJcjklAcJVFEtlVLtvFF0oevCP1l0p6WlJJx+DxJpXANlebey8L4+If14dfP5nCFCgI+l/MlTDSUOMqCXVvma4SW2CNzmtJxnP/JGB8bnR72c63Gzdm7StJVXMyDrFLnkkgC8RyctzPTPFnrSGURYxb4uhEp69HsIk/sc2RSIRHVRaLjJsL+ieK8jm1tHIoXNoQRrgmeTLLy8DwdsU5I02ciokgYkrdauD46yvxeK+wbQMMwDMMwjD7DNoCGYRiGYRh9hm0ADcMwDMMw+ow11QB6QpMEZqSEKhNP0zEI3ZvijUhpaYaqGF4mxSE3UoRD4uX07TZ2iTvCNYDZPBpMUpbrBirKS6+bol1N5QXbjao8ho1Pp3nDYoo+oym0BHPzaAi8UOF1VnqH0sPC5DWuaDaFXsVroebGrYqX3MdWo/hD9h/ghsmxZAFihvPczHv28BMQk0wJnVAWdXltacpLim5R6FUjT9GLZPi5HUUr88jjj7ByqOjXsnneVjeO/Sz1Isk09nNb6DFnFZ2g0+HjNzOM/bxxnOudes0sxDhCuBTGFGNzMS8cRfNbF/rCWg1fYD8otI3dDuppViIlxgURUacjtWhYv7hYx7Q2SL2a1KERETnCLFo7jyt0Q5rGTeoCI0Xfp+nn4Fri+lp9JJrWqd2WBtd4nnRSjlU015UGyVofSjQddSTMfbuKdtfzxP0iRefV5eNwYhUawGIJ9cZLS1yD6CmPZqnjjMWwXckkn4OVRXzGdDq8zuksnichrh8pftc58bIDTYteFs/BhTI+FzNFvl5ns7iOxGO8Pppms9rm4265jvM/nhFruKKjjoQQV5FIki+eDWFH0fcJ/XOoGEEnxPqcVIzfMysP8VPGvgE0DMMwDMPoM2wDaBiGYRiG0WfYBtAwDMMwDKPPsA2gYRiGYRhGn7GmSSC5NN9PdiM0hg0iYdKr6JCbQpCr6OopnuDizUQCRfyxOBd0y88QEXlCdNlYRtHsYrbMyqXRMYgpV7gotTAwBDGZJG/7cRCXE4UhP0+7heLS1Qixm3Vu9um38VopIayNZ1F0nSvyPssoSt+ozEXFsSrei8GAC3s3bxyBmGOr0OxnM/yeTkxNQkxSiOIXpg9AjC/E9tJ8nIiokObJPl3FTNtv8WMNxSTcEaaqjovJJI0Gvz+ZNBqS79hxLitXlzERIp/h/Tx45ukQ44pJV63hmN8otOwbR8ch5tB8hZUff+QhiKk1+fgN49j2QpG3VSZUEBEtCGPsag37+YpLn83K6TwmdKxER0kckUkgmoG7NF6WZSJMitOSMNyIz69EDBe/hFjHtFwOMNNWjHPlMqInnLgiRrmWOHVAqFp3A/Fdg/LVg+sJg2tlnninYO6dTOK6FolK95S1GAy2lY7uKWbVK5GKYyfmRBJas47nDUQd03FslzTY7ikZDPkiT0LpyRtIRMt1njSYVxIzxgZ50tVICdfQRkskGrpY55hIoAxjmGzTc3kd08q1klV+zFeSJzyR4OEpyaOReEmBZtztiAGsJQhJI3Nt/xGIxUSadBMRtRUz77XCvgE0DMMwDMPoM2wDaBiGYRiG0WfYBtAwDMMwDKPPWFMN4NioMGdWXg5frnO74Q7KmCgQhrsx+RZsIkoLmZnnoo2xNLykEGMc8cLoah3rPL+4xMpjE2icuyxe5i1fbP7DOnItQa+Lv+2nUrxhgaIpa9R5u0KlXTGhp9m4DjWJcaGbdBQ9hO/yOnabikahxa+V8FF7NTbMNSSnbdoEMcd2wSGglOP3q1bCa6WFDi6umFfn8iImieeZ2DDFypWyogEUmrGWoiVyhDFuTDEfjQm386HCAMRcfOFFrHxg3xGIWZieYWU3gfqVjnjJfaXVhBjyuL4nqZjFxppcz7NQQbPx6eP8ZfB+hGNeatqk6SsRUaPL+9BJYH16Pm+Hq7wMfiWCAOsH2j3FCFrKw0LlPIHQZ4FOj1CTTEqM/JzUsxEReFXH4vi3vufJY3gerY4QIz7nKO+uTwrTYO28Ul/oKS8SiHtSk6jpoYW+N6a9bIDXJxbheWLCtNxTtGAhKS7BK1CroolxMc91sMkYriOtOjdRjpS51BE65XQKNXdxoR1sd7ENrjDqLxRxvqXFmHKVW9EQev6KsidotPjzbGgI176ueA6mFYN7udaFmqZV6Do9xSTcCUW7tK/JRNdrUzAU19JmUjcQz27lYunMU1/HVot9A2gYhmEYhtFn2AbQMAzDMAyjz7ANoGEYhmEYRp9hG0DDMAzDMIw+Y02TQMYnuFgxHeLpnWkuCp1bxD1ou8XPE4YoZHU9keDhoCA21hUmz9KMlIja4jTNNiZduB4/txdiIkRCJBHUy3WIqTX4xaII6zNQ4iaduTSKVJvCfDgIUMSbTPE+TCVRSOoKpbhmcBtIUXoX72nQ4/cnncD7tXFqlJUHimh0vBrmjh9j5fmZeYjxRNKHr2QaDYxs4J/RjKAHuBj58DG8li9NghU1sCfMz3s9rE/M459LxLE+HZGAk89h4srBKh93hw7vh5jhkWFWPu20MyGmLgxbdx2cg5iGUDWfcd55ELPlTD42/Z5ibC7Mah1FTe5LQX4C2+6JZLGeMi9WIqGYUIciw6Pr41oDyT+aWbRMJlETPLgo3FcSTmIiSUZLkvNEfwWBci1RR1k/IsglUQ4QRUJsH0Z4rUh0iKskVEiBfkwZB644hoks2A4tOcEVdc5m0bw+IZPHHMXg2sN5uhLtNp4nGef3faCE6/7oMJ+TrSY+q+IygVK5F02xHmaw6TSS5klyjrKuOV2edLW8hOtaIAbMkvJcdMU+oav0jyPGeEcx4HZjoSgrRuti7iRT2HhPzJVOiNeKRL86Sj8Hwny9oxg6+2INyCjm9aGWYbJG2DeAhmEYhmEYfYZtAA3DMAzDMPoM2wAahmEYhmH0GWuqAcwVuH6mtYganOKQMENNKZqpJa6HaDa038659iIWYFPkz/KBYs7cEgaTnQCNKrPCdLbdQu1Fs83P3VV+7w993lbtZfDthniBdB61IKkUb3u3iyKXVpPrMeQL7YmIEsIMVeojiIhc8TdCLIaal1iSxyRzWOfBsRFWrtdO7QXXcZfXMa68hHtpepaVPcUwdc/uQ6w8NLkOYkrlCisf3LcbYgJhvBpTDKWTwpg6p2j30hlhpp3Bl68vlrkh+fzcEsTEUlwrMzYxBjFFoW2cWI8xQ1PimKJx8epck7hh4yTEhJ7QLSkvuacm78NOhPOLhEl5JovjUGqSkslT0M4oY0XqxcD1mVCbptkTQ4yiN3RWUWXwt1U+hDo8RSsnarmy5TM9ibaRlzUDZ6mjlBoqIqJAaEG1+iSFLi/uof4xJg14FVPuntCQKdJvchy+jknDciIi5xS+Q0mlcP53hRmy5+A8iWX59ZOKyXMxJdYN5UUC5PGeVXzpKR3nfRj18NkZdHi/+kpfxIVh89Agmjw3lvm5lxWdYCrPKynXOSKiRJKPDe3OdIRxf9DA80hNq+8ra5bQCcq5TYRG4jhxiWIkTae1nAjlHq4R9g2gYRiGYRhGn2EbQMMwDMMwjD7DNoCGYRiGYRh9hm0ADcMwDMMw+ow1TQJxhVA7mUV1qcxpcIsoLo0Lb8baMu5Tuy1+opirGMM6/FjXRTGlF/HrJxwU1saFkN11McmhJ0TNqnBUoO6+g+6PK/6wjkI8noxhP7caXFhfb2JyS1IIWTMZFNZ7wtS5K42hiWi5wwXelRYaZW+u8L5vKebMREXlGGfT5s2sPLEOExjKiwusvLCwADHTc2VW7iqy/QP7uIny4vw0xPht0VZPGT9C9J3L5yAmLQy/88romD7Ok1uOHD4CMYNFPnk2bZ6AmECYj9arixCTcHnSjqcY99Z6fEw1Kk2IOTJTYeW5abxWt7rMyr0Ik8fOesZWVi7k0Eg8EAbt7fqq0hoYoWLyLBMqXA+XzbgjzGw7uNb4IskhmcL5lhDCfk9JcgjF/ZNlIhSTJxRzdmmirBnOynNriWuYrqEkDAmRvOsqCRWiztq9kEbiXSW5TZpOey72oUwmiblKMqKYglrbk4rB/kpks5jglUvzNcIhfC66nkw8wHEYE88GWSYiiokxFmovUZBjPqHcd5Fk1evgeYKI3/dcvgAxA3leHwdvF/VCcd9DnF9FYV4tn4FERBQX+4aMYrwsmtpREjM6ImmnpyTJOCLpI5XEfUO3K14yoTxfXSX5aK2wbwANwzAMwzD6DNsAGoZhGIZh9Bm2ATQMwzAMw+gz1lQD2Gjw00Uuvmg5neUmjzH8CZ6S4jfvXA71R406/829UatCTL0mNEEtRRMUiJeCx1GjEBdikEB5EbV8cXlc2VrHhEbSUX7vT2fEy7yVE4EmKYb9kxHGmZ0yau7awshXkcFQQrxcvNxCnceu/dyQuDyH92KoxMfCyMRTf4k6Eepnsoqp8uBAiZWlbpCIqCv0j+Uq6haPz3K9WuLcMyCmssh1efIzRETLNa5xqy5XIEZ2fbZQgpj8PNct+opAdObIXlZWvEcpLXQvZUXbuHCY6x+zKdSvSN1ioYhz58HvPMjK3/vuQxATCbPa8bEhiLn4An4PY4rBbeTzY+HqrI0Zsk1EeG9ait6nK45pujypCZIaPI1uV3sRPa9RXNF5Sc2fZs4sDa69uGKKCwbOmk5QtF01i17Z8FYOVWlUT0REoq1aP2vHJKBpVdrlKy8OkIABsPI8kywtLcOxYo6va+m0ot2TT+sIJ7cvXjYgdadEROTxGK2fXXFPox6exxP3IuphH3aFjlOpMiVSQo+p6Cr9utDPJvD54Yq2ZtJ4M3zxzO0q7fJhAGOl5fjV5oWc3zHlee+JexEEipZQ1d2uDfYNoGEYhmEYRp9hG0DDMAzDMIw+wzaAhmEYhmEYfYZtAA3DMAzDMPoMJ9KdPYFbb731P7gqhmEYhmEYxr+H1e7X7BtAwzAMwzCMPsM2gIZhGIZhGH2GbQANwzAMwzD6DNsAGoZhGIZh9Bm2ATQMwzAMw+gzbANoGIZhGIbRZ9gG0DAMwzAMo8+wDaBhGIZhGEafEVvLk335rvtYuVKpQEzSDVl5IIE+1BuG0qw8MpiBmKFSjpUTHjYlluTnIc+DmKXyMit3fQihgVKBld2gBzGdToeV2+0OxKTSSVYOKISYZqvOysVSHisU8c91O10I8SjOyx7u9fM53ofZLPZzPM7r3Opg2yNHnNvFfu6Kz/mRAzFLXgGOScafdSUrH33iQYhZOLCblYMAx8bohu2svGHraRAzMD7Fyqk0nmf3o99l5cN7H4WYXq3Byp5Sn/wAb3sslYaYiy67lJW3nrYdYtrLZVZ+7JEfQEwY8nvR7bUh5vFHH2flamURYjpdPu56XRxjS4stVm40cV74AT82PDIAMQODWVYOojrE+GJotlt8bXnWuefAZyS/des74VgY4jz9L4nySgDH4fO01WhBzNISHxsDgyWICbt8jKUyuNZ4iQSvjlxXiCgkXh9caZ5e/vB3f2/FmL/58F/DsXSaz3fZ70REMZf3h+ti//hhwA8o56ks11g55SYgJuvyNarWwfvuZvizIZ2MQ0w2y+dtoYhrfKVcYeVuA9cIOTR72oNaNNWLYf8k4vxYIZuCmImRIisfn52HmEaX93O+UIKYoMdr3WhUIWbdOv58j8eVfUyMH9tx/vMg5lSxbwANwzAMwzD6DNsAGoZhGIZh9Bm2ATQMwzAMw+gz1lQD+Ohju1h5eQF1QwPiJ3dnKAkxwwHXpjnpEYhphBVWrgcoYIkcrklotVG/1mxxvUEvQL3PosfFBakYXsv3+TFP0cElk7ytzXYTzxNyXZXTHoQYeepeBzUT6Rjv6LqiEywHXEeRzmQhxhFaEMdDnQcJLUpT6eegx4+5MbzvqTOejecWVMtc4zZUQr1YNDLKyzHUUU5s2MTrF6KmxA257iVsYky7vMSv1cJ7MTk8zMob1m+BmKlt6/ln1k1CzOgonwexOPZhUOJaovVToxDjC7Fcu436nkqZ6xYXFsoQE0uI6zs45gfE/E5l8VrLVa7DTaZwWQoj3vfxGOqWqstcY9PtKKK2U0DTWvUr3SbqmJaOHmTlI4/XIGa5ysfTZc+7AmLyaTmesd8dIfT6abwzcUWLHog5GSrPIUdoJDs+rkdeTJxb0QCW8lx/WVC0390afzaFLVzTM3G+1hQzqFtOi3uaS+DzY1GsmWGEz6pUird9eASfi1JLmEor+r4JvhZ7ivB1dJQ/U+LKeQ4emWHlRBz7OVPi/ZrFxysNCU2kHN9ERI0m7hPWip/G+WMYhmEYhmH8O7ANoGEYhmEYRp9hG0DDMAzDMIw+wzaAhmEYhmEYfcaaJoGATy5q1GmjEIVvHENjyFFhBKuZhkqjzLaSCNHucTFppAhiE8KAk3wlmSTk5y4oxtS+MH1MxFE4GgiPTjeBQvZul1/L97HOafG5mGJmmRICfd9pQIwrDKV9RYAq8l8ol0Whb12Yw8okAyIiV5ynXkUxObZCQSSTSINpIqJWk9/3jadNQEy9wfuj28PxMzjMk0dicfx7afv2baz87EueCTHrxnhCR7E4BDG9GB8cmRROHk8MTUcRgbdEuzo9RbwtxnyphElWW7eczspPPL4XYsjh5+4oZrFFYZAaxyFPy1VutBoRisDDkIvHy2Uczy1hMh2tTQ4IRWt1op9wtHa6Dj82c+QQxDz8rW+xcq+F4yCe4+a6LWX+54WBtDR9JkJz6J/GO5NQDIod0a6BYUxyaLR4MkA8UIygfb6OOMo9HR/niRDjSkLFwb0HWHk4hs/psUmeYOb6WB9XPHMLkOhDNFjkSZ+RF0BMscjX4oySuOK5fD0cGcN1NiWSUGrKOPQjvq4VS9j2SZn0qeykYnEek/Sw7aEwlC7kMWEx6v3HGdHbN4CGYRiGYRh9hm0ADcMwDMMw+gzbABqGYRiGYfQZa6oBTDniBcl5PP32dSVWHkqjKWZcaO4aS6gJCkK+d201UTcg33FdKKETo3wB+fIyvmRevIuZBvOog6tVuT6jq5g8t4VBcqRoXLJC29ATL1EnInID3mfxpGIIHPBrxaSYj4g6Ha6ZSMTRpNMVBsmdegViSJhwJ5Q3tAchj1lWXvg9DEcQv837w/FRH5FIcDWhZkg+NM51eRvORnPmkfXjrBzXBGxC79jzsV27prlZdHP/AsT0XD7Gdz38KMRceCbX5V1+EeoNpY6rqmhcDh+SJqY4fuIJrkUZHkEd5eEj+/l5Uoo+VOiWqlW8FzFholoo4HlaQlcWKO+C98VYSCaV+3UKSL3xf1UiwrnUE7rO40eOQEwhw+dbupSDmPkyX1cXp6chZnT9On5AMdOXijZHiot/CigWsH+SQhsnTd+JiOYW+TqSUtb9apmbqo8Oow4umeT9mgbxPtHk+jFWls8lIqJeVzw/COdbUjxfmy18nq2f5G2N4sqanuTPpm4Xtc3DQ1yrF3PxPJ0O1w7ntbWmw+tYW16GmE6H7zeGhvGeprO8X2MO7lFiXd4/7Qb2j6/o3NcK+wbQMAzDMAyjz7ANoGEYhmEYRp9hG0DDMAzDMIw+wzaAhmEYhmEYfcaaJoEMSHGpIsIuCiPh4QJWIQi5WFKzQfRiQiDs4l62IxIY4DNEFBNmyEEHRZiRx889N4ei0LDH61xrohlqM+BC/1waTR9JiEs9pfXSnNVTxMAtkWSRieO1YiJhQCapEBG1e7wPQ8V6tVLn16o08Tz1Jj9Pu4f3ayscQTpNnlSQU4xFC4M8neSZz9gBMeu38KvVFFPl3fuPsXJVuaf1Ch8LS5UliJme4TEFxQiaRBLI5/7pXyAk/kreZ8999iUYE+d9Pz4+DjEU8TpWypj49MADj7GypySKZIVpqR/g2OjWeds95U/OEWH8LhOYiIgWl3jyiEsoSo+JbK2SYuBq/D9kwpBcV4iIFpb4WDl08CjEdJbKrJxPYTJZs15j5V0P/QBixjdtYuXiOCYeSXdvzaP7Jz1pZ0gxeQ5Dvs532/gcGhvnyRKZFFrnJ8UEmxjB1Lpej69jiwuYlJYXiSqaCX7Y5XWOx7DfXZffoFazBjEyF9JN4XO6022JMiaGyqSvehXXtWyOrxuBfDsDES0uVfh545g8KoeYfIEDEVGtzhNOXCXps1vl1+928TmUUxJw1gr7BtAwDMMwDKPPsA2gYRiGYRhGn2EbQMMwDMMwjD5jTTWAwyWuScjH8bf8VIrvOV35hnsiSouX1fd8/J1evig8ilAT0BUvbA6U39dD8eLnKMDzRDGhLeiiyXMgzJmbAWr3fHGs1sD6HFsSL/xWzCwLdd723gzqzlrL/DwbhhWj41Fuhuzk0TS4W+b6nnod275c43qVhWXUrxw6ws8dKG/Pfi4cQZJJ/rmehwacrTTXbBysYn0e+sb9rFxebEDMseNzrBxXzLTl/en4io6yzcfUxAi2fW6GG+wWkqijqle4fmb3gUMQMzHB9UXxOF5rYj1/ifukKBMRHZnh+sddDx+DmJEJrmU8dBjHIYkXmUvdEBFREOPzO5VQDGVjvD9abVwTCgWuSYzF1sYI+r8uUk+HfXrs6HFWPnD4OMQc3cvH4XAeNVPrhrmOafowGko/fN8DrHz+lUWIyRSErvMnW+6n4iq67q7QngddbJgv9HQd5WUDMaEBrFbKEOMQv8+RooM7Nj3LysUcrrMZMSerHXx+SJ1pQtH39YT+uqfo6Ryh8Q+1PYEnjOATuIZKCXuzhc/7RJKP1YTyAoBMit8fzXR+WejDqxXUP2ZTfM1yFJE0jPk1xL4BNAzDMAzD6DNsA2gYhmEYhtFn2AbQMAzDMAyjz7ANoGEYhmEYRp+xpkkgkyNcPFlIYJJDNsPFkk6Eonmp1HQiFM12WtwYUjNZHMxz8WQ2i2a21WV+nmIBDZNrbd6OQ8cWIabe4eLWhOJevS7DY2JxNBY+uMiFo91IMa8Whq3FAgp0n33WeaxcnUbRbNQU5xlG0Wynya9fr2M/J+P8c+vHsT6jo9zEdLaKQt/VkMnwhIW5Co6xfUd4wsLjjz4OMa5Ijgg6eJ5WjSeGeEpCTqvDhb3LNRT61hr8PIeO7oaYTJr32Wlbt0MM+VywfO/XvwEhGzdvZOXtp6O99tAQnxfJFC4DxQKfp66PAu9mh//92GriPW1VuBlrEGBMKs3Hj2bgWhCm00lFTN7t8rWkqRh3nxryvq8m8+AUsxMiWVScjuV6qBgfO6v6255/LgxxjeiJpKa6co+PzvJEg9lZTDwIAm5IvG4U67frezwxa3R8DGK2X/gscQTHrhvxdin+1vDVR6TcLu25sxY4yj1NJHg7ZPIEEZEvDNI7bVyzBtL8GRx3sWExl8+3dhfnUkK8XKDbUZIsqzwJJZFDY+qESOhylMTQQKxracXguicSOPMFTBBKpXidHQfvnzRn7nVxzDsi6SOl1Id64l4o8yLo8kEWj2FyVGGQm+D3enhPq421WscQ+wbQMAzDMAyjz7ANoGEYhmEYRp9hG0DDMAzDMIw+Y001gIN5/lt5rIu6oaTQXmWSaYjptIQxZIi/i5dKJVbWNBPdgO9vez38nT6T47/LT89jzL5DvB0LNaxPUxzakEatw0svP4eV102gJuBT3+emqt/eOwsxfsj1BzEX216rcJ1iq47tyueF5i9AvUhKvNhdM/LMODzGV4xF168f59deQp3XaigNcqPjfUf2Qcz0Qd6HmTjqV6qNCivXq/hCdEe8oL1SQ7PoSosbuMYUA+fhMa5/SuXR2HPdph2svD6Ff5sdeOh7rOw5qJ/tCbPx+QU0Z95xzumsvG37RoiZmuB1zl5yDsQ8/MRRVu60UWPbiQsjaEKNbRjxyTMzswwxUpNUHMCX3EsBXauFBuCnhiYiW+kTq9AAaqeNpDkzBkXE+0vV+wldoKPUZzVHNmzawMqZvKKRboh+drA+jxzh8ysdw7ESE4bpj917L8QMreNa4tLUJohxfKkhx3bJ+xMqa6hyaE1wXeyfKOQXS2dRd9YWmrZENgMxQUOsdYoAcmyM96G/qDRU6PKyCWVui/WwOD4AMc0mmlVLhse4oXynjuuaJ54x8TjWJyX2Eu0WrtfJBO9XN4HP4OUGf1b2evg88wI+B9uKHpNC/qzUtI0xoZFs97DtCwv4bFor7BtAwzAMwzCMPsM2gIZhGIZhGH2GbQANwzAMwzD6DNsAGoZhGIZh9BlrmgQyIgT6rSVMPHAdYSwssyeIqC1MHz0HEw+aQpip7WTbPS5kLQ6ggLkbcAHs/qMzEFOu8mtFsQTEeB6vQSGFwtGRGE98SC1hcsL2Ajc6nhlEAfNshYtCO008z4O797Ky66MpZi8r+qM4CjHk8iFSKGLSTk6cutNFIWvU5QbJG0dQwKzIaIF9+7hZ7BP7DkDM9DQ/FtRQiJwv8uuftn09xJx95pmsPDOPSQWH5vm5R8ZHIGbDVp5kkR/CBIa5Mj9PtHAIYo4c4gbX8xU03D3zLF6++rTTIKZR58aiiv8vRV0hyP/29yBm2+k8MWRsHZqzfue7XMg/M4tG2dL8tN3C8VMp87mTzuG1QmHc22iiCPzUeOp/J6vmwwItwYNEMkAYrWzOnEhg4pEDFdASIWQItrM0wAX6lz33Moh5+ME9rHzowBGICXzejr3eHMQkN/FEsWAXJniV/u27rHzRi3EupTNc2K/ktoF3tpay468i+UdLrlmJ4/OYHCnHQqaD63VOrFltxcQ45/HkiHUTJYhJZnidPVxGaEC8sKGUwaSL3DgfG10la2b3zDQ/TwmfwZ0GT/pqK3uCuGhXr6rsGzp8vxEq+wYvLl9sgMmIvvBdlnsEIqKREr8XgwXsnz21g6w8NFCCGFnFgpL8E/bwxQprhX0DaBiGYRiG0WfYBtAwDMMwDKPPsA2gYRiGYRhGn7GmGsCBYa7HGFBeDu2KF1FXqhWI6TW4dsdVjIVDoc+IlJdMZ4VOqEf4O/0T+7nOpNFRzCPFS6aLCey2dJZr4wY81Cjcv3eelX3lJdydIn8B+sgA9qEjzHR7Pmotm10uZGg0FaNsoctxeqgllBIX7eXikStfeo3t8sXLxCNFV7EavnPP3awcG9sGMVuEEC7dRT3NGWdtZeXTT5uEmKDN2xW5qAFsEDfcjsXxfnmeGIc+jsNGjQtxioq+xxd9dmSuAjGp3HFWLhRKELNlK9c7Rsrfga0KHz+7vvMIxEQt3q9nX/t8iNlx7iZ+3vtQA7hv72FWzmTQnLVYkiaz2D9VsZZ0O2v0EnVpJLwa2Zdq4CwMipWP+cIUe+/e/RDTanG96Olnos4zmeRz0JWiN4UwwnkbEr/Hz77s2RBz+ADXef3tX/49xPhC13lkHs2+k0J3tm0Q9X27v/4AK49MoYn56Zc9i5VbyliJhbw/Ekr/lJtcq9fp4vootY2roaPosZeW+PzPNHGtGRTrc1x5fKdyQifYxPkGuntlaHg+j+nUsO3Dea5N270Hdcu5FK9PLq28+EHM0wFFt+gE4mUDiu49Jbqj1sZ7kxSG8jOzqPmnkK/huSIa97dbvM6+YuCcFi9NyGdRq7skzLSljpGIKJ/D9XCtsG8ADcMwDMMw+gzbABqGYRiGYfQZtgE0DMMwDMPoM2wDaBiGYRiG0WesaRKINA124ih6lCRTGJMhLhyNKftUVyQe9AiFtck0F28uzqDpY3OhwspbBlHE3xF63FQWhaynbZ3g9ZMfIiLf422VonUiopjHhce5BBomDw5sZuWt29dBzIHDXCy9a/c0xCRiXHAaRZgA4/v8nroxvF9xYUQbhngvQqE0dhTT2dUwf4QnXZz3jGsgJpnkBqUDqG2niUmeSLNUwbFxdC8XZndDTN5wHd5WL4ZtDyIh7PWxQoEYL1GA58kWudH6Yh0Nrt0EFwyHmtmwNLjFS1Euxftn4ySOMaFxJpdw/OzYsYGVSyU0cG637mLlmekKxKwbFSbBDoql43GeQFWtogD+VJB9qJk8SyPfKMAkMBjySuLB0WNHWfmzn/8SxFSrfI24dGEeYq644rmsnEzitWS7lGFAgUg8yubRlPb6l17LyvsUA+evfuFuVq72sH92HePm0AMOrrMpkZj1nS/eBTGxIT4H3DEcc40K78O44oY+XeUJVdUajqd2G9f5lRgZxD7023zu5HO41kTCANyN4RqaTvNEGm36N0VCTtfHsZEUGRVnnL4FYmZm+P3qdPBiwyN8LfaVeRESf36klaSHXlOss2mss+fye9hYQsPtZZHYUyygMXVdJEwGIdY5KfY2PR9j1m3ga6Z8BhIRlau8D7VnZ2lwCI6tFfYNoGEYhmEYRp9hG0DDMAzDMIw+wzaAhmEYhmEYfcaaagBbbf47uNPT9BE8ptFAXUW3x/elvqsY5za5ZqvaRA3X5HrevMjHmI3D/Hf5LZPYJc02j1l32tkQkxA6r/KyYgxZ4houWkQt2PpxrmOqNFDntfkMrscoDKBWJj/AzWHL89j2yrLQwSh6QzcSL+FWtDJSthAo+h7pHy01U6slk+OGwHHlNJXKAisnB1ED1BRmrJqUJzXA9SFJTSQlzEYjZUZ1etw0NJnGINfh4yV0cWzkhrgOLhFVIMZL87ZGCfwbL3R4fZwAtSmux+sYV0xM0zlhztrBMbZ4jOvTBrOoZ3nJC5/Hyvc9dBhiGi1u/NruLEJMp8VvYimP9/3UEGNe0a9WykusvFyuQIzj8X6emUft3rfvu5+V73/0UYipLnET5a5iQnvWOWey8sgI9rvn8TFWq6FxdqXCr7VxCg3TJ6a4YfMvvOFGiDly7AArf/ehxyCm0+Drxp6jsxCTGed9v/TI4xDT/DQvb73sGRBTrvOx2lQMkzuO7GfUnYbhU1/Hckmc22dsnWLldAbXdDknZ46gibEvXgqQyY1ATKXO54nnJCDGEXq12jL2z8IcH/PKMCQS+r56HXXCYcQ/2GpiTL3K21XIoI6yK/YWkYPPIU/kDhQUTWs6w+9PTHmxQT7PcwXkeYlQz3fg8HGIcWK87xMeXqvWxHG3Vtg3gIZhGIZhGH2GbQANwzAMwzD6DNsAGoZhGIZh9Bm2ATQMwzAMw+gz1jQJJHCEIF4xfZTi/3QKjZdzeZ6MMD2P4uQDR7kIPKZkAyRmuUi2PbsAMdtGuUj1eVduhZj9x7ghcG7dMMQMD42y8pwi8C6VeLvcEIX1CSH+n5tHA+dYiouT5ysolj4+zYW08TiKiosFLlJttbAPI2E26shsDiIKRWKIqxjcOkIkG5xaDgiNb9j4Y89LRNRuc8HybBWHeUIk5PR8RQgtzD5bioC5F/Hrx2J4Ht/jxwYU89GRIX5PoyUc890e72cn1Ixg+X1WckkojPi8DAJM7HHj/NyRh9eqN7iQ3lFMTJPi/tSUeZHK8MSey5+NSVa79x1h5Ucfm4OYepXfn0Qc15aV0YT+ol04vGm5ygXx37j32xBz6DgXgS9WlyGmLPrUVZJvUh2+jswtYkLMN+79Fitv3IRG3skkT/A6fhTXx15XCvSxzvWaSCZTnipnXLiJlR/ai8kb3RpfFI5VMPEgk+B1nipiguDB+37Ayl5SeZHAJB9zyz4m28HUiXBudzp8vChDA8glcFJmM/yexhPYiYUSr7PihUzlRT4OH3scTbn9kH8wmcBEiMEsT6CaPobPocUFPu7aPt6L6rJIDFMSqCIxvWTiERFRj+eAUbfThZiMSN4YHCpAjHwBQcfHtS8SiT0tJUMwIn59XzGClmMjUBIo0xlMvJTE4jju1gr7BtAwDMMwDKPPsA2gYRiGYRhGn2EbQMMwDMMwjD5jTTWApRJ/ibMfw9/F63X+u3jUw9/Fl4Wm5NBhRe8jjDzTKdzLTh/gGpKxFOpp1q2bYOXS5AaIidWESEE5z9QznsVDZlAzkfa5ZiIg1BY0Grx/JjJo4NoNeH2cLL48e12WmwbnS6MQU1vk/To3uwQxPYcPkXYXtRfkihfGJ1F71W0JTWIC+xBHAhI5XOfRU0ynmzU+NpJprE+tWuH1a6P2q1nl54krmpt8luszhgcGIKYwyO/PSAnrE8S4XqWVxHaVN/J72glQ+0nCdDrw0Z01FBqgwEVBphPn/VwaLOF5AnEtZS4Xi7ytCQevtVwT+sceai2fcSYfv6U86mI+99m7WHl+VmjjUFoIPPY4Gi/HYnwOSF0cEVFZ6JYqdXwR/ZFpviYURwchZrDINZyDwzj/F/bx+/74I2iq/JWv3M2vVUANsCcMbjtdvDfdDl+jvvQlnCdCLgrG0EREmWE+3889bzvEPPiNvazcJNSU7hFrVjrAtW/A55q2fd9+CGIqI1yvtuTiteJdPnZ9ba1p8jnw3HPOgBjJ1Dj2j9SHlUoliPHE2hcfRqPzsRG+/tx1170QE4b8PMU8Lmwz0/y+jw3gmlUq8r6vzOHzbGGOj9XSAOqfs2INLSrXymV5u/JF1PdlpTF9S8kd2HuUlb0YPoeaHT6/u8ozr9vh98tTNNIOyXwHXLMCh1+/p7hp9zraCzXWBvsG0DAMwzAMo8+wDaBhGIZhGEafYRtAwzAMwzCMPsM2gIZhGIZhGH3GmiaB1Co8iSDWrUNMXBpBKka1MY8fbCqC6oE8F6CWsmhC2SrzJJCRSRRdT557GSs/chQFn3v28mPPnihBTKXCxZtjW3dAjENclNrroPFqSbhiVufKEJMWIvSJQUw8qARccBo/twQxLWEgfe/nvwAxR49wIb2nJG9I+1PFT5p6IsZVxK6rwuf3Ihbi/RJ5B7S+iCLn07dwEXEupYjkxVhtKMa97SYfm+kstuu07SVen42TEOPG17NyXTFDXT8xxs97AMdPYZDPg0FFdC3NqkPlfkViXqay2D9+m4vilVwSigsj6LZitDw4zOdyvYni7UaFi/8nRzA54qUvvoqV/+/n/g0rtALf+u534Viryk2Csyk0b73+Rdexsq+YBt//8B5WLuRRxN8Oef9Mjo5ATG+Wi8KrDeyv1h6eUFFSzJCzRd6O3AAmJ6SyfD0qlnDBLghj80IBEzPSOX6tK553PsQsL/C59OgjByEm6PG5fKSiJKUIJ+rYDCZv1Mr8mJ/H8e2m+fPi+BFMuqqKsUGrSAKJpPMxESXEuqolFfQa/FpJ5dkZiUy1QDGLd13eP+q3QCHvnw0bxyBkWMzBddOYvJVM8nYVijh3PI/XeW4OEyiffTFPshyfHIcYP+JjoaoYpJcX+Lq6VME6xzy+kI0MY8JJKBbNMMB7WszxeVCWpthEFIkXK3RbOJ4DJflorbBvAA3DMAzDMPoM2wAahmEYhmH0GbYBNAzDMAzD6DPWVAMofsqnoIW/r0dSC0b4+3YgDC/LilysWuW/wUcdDJoQeoMLrnoOxEydfgEr//NHdkLMuDBa9rpozHhs/wFWHttyOsSkhjbzAxFqAppLXNeVDksQ0xUGlws11ACVRjay8uA4Gly361y746LUgYIEb6vjop6uJ97U7Sgv2HaE7sX3FR0MXh547rOfwcpbzkKt5XHx4vJ1k6iR3L59EyuPjaDWyot4W+s11OV1elyXo/VPToyfbA6NTr0EPxZXtI2tBte0PHPHeojZdBq/z70Q50Uk/u7zQ5yDkZjMXhyXil5b6GAUrYob49dyUoqbtojpKvpQz+NaoqCL92J4mBsAX3b5M1m5Ul7ZavzAwYNwbHmuwsrbNm+FmHSarzXTx1GfeegAN6HNZXEcdISRt1PFtaZVEf2sjLmtWzex8pYRnNx5oQ+dn0OtdWmQ35uJ9ajhqlV5nRM4tSkltGgFRcN59Qv4+lwWGm4iotmj86y80MGLpZf550YLig5WGJKvy5cgJjvGdW/HDx6GmG4T1/CVOHzkOBzLZXm/1mtNiCkKPV1Xe3YK0/J0PgcxvRb/3OhICWKSLh93W7dMQEwiyVdsN446ykRS1CeNGnJXjN+ohX3aqfK9RK+I82Jogo9x18eYjeu5/jqZwjFWbfC1JZFAsWVMvCDB19YsYbQeKHsUT+iJIx81gLlsCY6tFfYNoGEYhmEYRp9hG0DDMAzDMIw+wzaAhmEYhmEYfYZtAA3DMAzDMPqMNU0CEbpaChRhpCOMYWPKFjRq8c85iqh4cIgLTsczKIh95gXbWPnMSy+AmPIcF5cmfRRCb57iwtFQqdD4KDdRlSa5RERNYRbdU5Ilei1+SwJCU9V9x7iI+JFHHoSYZ1/CkwgGx9EEu1rjQvU46rtpeBO/fujiDQu6vB2+InatznNhbaemCIbx8sCzzj2Nlc9+JiaBtHbwZJtsEUXgcAcdFNK7Hr8XA9lRiIlEd2h/UYWhSIDRjD3FXOl0UMC8ZdsUK6cTeMNaDT5+I1dxixUC5khOXCIKI34sUPpHmqHK5CQioiDkdXRjSj+LXqst4nkOHeDGxpc+5xyIafa4oDsjEk4q8AmksYzzv9nm9yKZQdP55Rr/3KEjmDBQEuMwaKDg22nzeTs9sx9ipo9zw33HxfPc+IoXsXJYR0P5u77xTVY+9AM04B0qctH+7B68f5OTfFwu9+YhhuJ8rRkcwqSrHaefycrdn8XH00f/9ydYuVXDeTItzX1juLJ0unxO1hewfybF/UooCQzDo2jmvRLNFiZ4hSI5sqskyQ2IZI1QSd7qtPk6sn79Ooh5/BFuSB5X5uT4OE/SGVYSRTzxHIwr7wiQSSAZZe5II2hq4TrbrvK5vTSPSVaRmAdpJeFMXr+Qx7Wv2uRjIQowJi1eHOAoY0wmR+bT+MwLRN8XMnieuLKErxX2DaBhGIZhGEafYRtAwzAMwzCMPsM2gIZhGIZhGH3GmmoAQ6FbaHXwt/OEMLyMxbAKnst1DNvGSxCTSvO968aNqHU49zncWHT89LMh5qFv/SMrb1iP1xo/+yxWToxsgphYhptQttpogt2qcoPL2ePHIKY8y/V9YQ/1UKk81zEMD6P44ujxh1l5bAKNPP0mr2OkaFOcRoWVgwg1N1JDlk7iPU2M82PVpGIIvArSYvxkU6iZyGSEaCKGIgohXyNH0wCKY6HyEvewF4oYHPNS9+oTxrhCTxM5+LdZrsQNrX3lBeTw8vcQ2xUR12xKI9Yfnki8VF6Zp5Fsh69ofkN+rWSI9yIe8Dpn2xgTzQrz8/2oM1t3OtfqLro4B1eiq2gvmx1uyrvvwAGI+T//519Y+Zv/di/EOMJYfLaK9Vs4xNeEmKJ/9kWfJsbR5Pnee77Fyp3qEsQ8tofrKpuzqEmuzPNjpSHUcM3PcC1adRlNjAdKXP/UDfZBTHT3D1g5XUAD94FhrrVe6GG7mh1e5+OKTjAS609mGe+FJ3RmxSHsZ8976o9QqS0mIuoI7WdS0ZR1uzwmkcI1whXrUdDF50etzPWqzTqaIW/ewHXUaWW9zmW4yXRxAI3Nez4fG0GAzxjP4+0YGkbz6vk5fg9n5lGzef8jj7LyNqGZJiKam+dtPT6NWkKfuJawVMD6xIWKPJnEeeGL506njVpduTxnBnGM1epPfR1bLfYNoGEYhmEYRp9hG0DDMAzDMIw+wzaAhmEYhmEYfYZtAA3DMAzDMPqMNU0CiXtc9FipoRg4aHPVYyqDwlHP5eLykSE0vD06zY2Ftz7r+RAzdY48VoKYnqhjIY8izJHTuNlwI4bi5EcfeICVuy1se7XK67xw7CjEeEIkm0rhLZrczBM6zjltM8T4Hu+zuIeGpfEEF+3H2ooA/hA3hw0V82pf/BlR91DEnxH3cGwSjannMTcCyBd530ceJsC0hBF11EHhcUcca9TxfnXBnBmTHHyR+NRTzM/lsWYThdmtBk8Q8kNU/+eEQDhfxLFaynMD11QCxeRBKPrDQUNZl/ixfB7n6eKc6J8WipXDsMQvBRFEYcDF0fk81nnjBm4O22ri/YqEMW4xL03U0eRZUlRE2D0xvquKaP7xBx9h5bkDhyDGFcttOoZjN+7ytkddHLuu6MV1SoLXQL7EypWmYiy+aTsrHwoqELO8xJMsgiSuI7PC0LrVVJJJluZY2VHWiI7Dr19pYrKNm+DJJKGHYyVK8Os30fadAjFvswk06c0VS6wskxWIiMII27oS48NjcCwZF0kpSWxXKsNjfCWhIi6y2wopnNtb1nET7pLyDJ4QBte5JLa9kOWJD21XMfcXCUvVZaxPKsuvH8/gvJid52vLkSVcQ3fv5Ylhs3M45qsi2afXwzXrzDP5/cmlsD5BUyR0KMltkUgITCXwWS7HoaMkCPnBUx9jq8W+ATQMwzAMw+gzbANoGIZhGIbRZ9gG0DAMwzAMo89YUw1gpyVexqwYAjvCvDLuoiYgCvixdA73qS9+1QtY+dnXPRdi8sNc6zC3fzfEeOL68qXuRETzB7lh6vEa/ib/b//n86ycU14c3u5wvcH4WB5iCkK3dOAomkV3RZ0HJzdAzGnnnMcPBKgpWapw0+lmGxValRa/lhPhPW23xIvVFTPkqM7HxhklCCHCKgL/9zNfYeUgLnVeRJUy14LUlxchRshMQRNIRDQ7y88TSPdoIhoc4WNsYBj1oUmh62gsLUPM7j18bGrmn1Ob17OyF8d7UchzbeWmzWiGOrWe6+k2bUEN2aAwftV0MGFRjF9F19UTc9mL4Vz2xLXGNg1BTKrA9UY9RXsl5WADg3J+rawBzCoaQC/P9au9RdQfLuzm83Qqh+dxhL6v1kKNUsfl7XLSaDCbcHg/z88qprjf4ZrEsTya2S6W+TistlBXVRfyudYC6h+lsjOm6PLScT532l3Uys5XeH0CV9ESx7jOTJqsExG5Kfk5xU074tdvNLDt1So/NjCE+kfNaH0lIsV4PZWWL0jAdsXFPGnX0Fi4J4ygi3l8xjzjPD6/5L0hIorH+XyPKcbUsB66OJ6TQveWy+E6khDtihQ9XUzc58ef2AsxjaYYUwHO006Hr0cJRUPuunzORcpLAkIxT7W5UxM6wZhyrW6X18dXjOi7HbzPa4V9A2gYhmEYhtFn2AbQMAzDMAyjz7ANoGEYhmEYRp9hG0DDMAzDMIw+Y02TQMJICOlDFGo7wvjQjzAJxHGEgWISjaCfcf55rJyIo8Dy8QcfZuXycTQW7QjRZa2Mguoje3exciNCw8u4MLPNxVDIWkjxdowMoFB8epYbpvo97J9mjScIHD2AhtJET7BSvV6HiFSM97OfHIGYRZ+LiNNpNA3NCJPgtCIYrjW5eNwPsV2r4St3fZeVS1PbICYKeP88eO83IGbD1CQrDw9h8sbxo+JeKOM5M8iF4V0XBeezIpHneRddCDHPOPcMVm4qwl83zsfUwcNHIGb3noOs/MgjP4CYYpEnzrz8hhdBzKVnb2XlRIRC6HUT61i5qySBOOJPTCWPhnrE+9WNYT8nS1yYnVbE/6HH1x9cEVYmTCh/Ewe87XHFEDje43XeUMDx5IukhpoiHHcL/N64CUwCac/yZJZORUneWOTzfSHEOi93+Oc2PutsrM88N4KulDGRJpfj61pbMenuxfka0e7gPW6JBAZXS5ZISIE+JpMExAeZp6zFrs9jQmVgzs/zpBTFA59iCVHHSYyRdHt4olqD91kxj8+8ckWYGPu4hqbTPNnHc3EWLC/ytbirJIFU6/y52AtKEBOJhAolJ43iYsw3AyWhQXRHt4UxGZFQOjMzAzGdiI+xjodjIyGeTR4kDBE1hZG5ryQsJRO8X5eVlyjMLlZYOSK8Fol11XFwbGjJtGuFfQNoGIZhGIbRZ9gG0DAMwzAMo8+wDaBhGIZhGEafscY/LnMNR+jjb+fy/eeBIqzoihfRj4mXchMR/etnvsTKg2NPQMyI1Cg1Ub8Sj3NNSS6LxpkxoWPIKGKH8VGu+WnVKhCT9vi1FoW+hoio1+X9kU+h5q4rTIL3PPAgxEw/sY+VOz7qhEhoyjTj1eyU0KJk8Z66Sa5/SCr6vhJx3eSZZ2+EmAf2wSHghte8il9rdCvENGuzrLz34UcgZmKcjw1Vb5TiY6EXYh9u37GFlQcmUEfZHC6x8vXXXQExUkfZUDSA0nPWj1Bv2Pb55+bmUNN6+ADXz2QyOOZnjvKxeejR/RDjtvm19s/MQ8xF15zHyhs2oem0NIt2FdNpiouXpmsaUqGfSTiKAfAKLFdQK9tpcm1htovzZHh8nJWXDi1AzN6DXLO50EPd0MAgX0dcZf43wgorBz0cu76os6a584XWen4GDdMbdT7mox7qxdJCo61puJwk1175bYxJZPl5okAZ38KwXZE2UtfnMck4apITKX4sl0Gj7JQ41lPa7ipa1JVYKKMR/KR4fkhNIBGRH/I+G1B0y/Uq/5yvrPudruhDRZf7xN5DrOwqc0lqYTdsGocYN8efee0GjsNA1MfvKobS4lrLihZ19zGuhd88MgYxA3muu4+BWTxRs8GfcRUfrxVLSD0vjueyOBZGOFYcoQuMKxrAhjCULkHEqWPfABqGYRiGYfQZtgE0DMMwDMPoM2wDaBiGYRiG0WfYBtAwDMMwDKPPWFsjaKFST8Rwf5mKCTGpIr6PPC4GDhUjxoUFLvSvz89BTLrHDS9DZb87MDDIyqXJIYjxAy5SPX4crxUJ81HXxa7tCuNOz0Gxe1aYRfuKjt2TBx1U8QZdLlx1ZQYBEVWbXIzcTaJgODfJ295Mo4C5FgrBeQP7eaiwiZdHByGGVpEEkhRGvbufeBzrs8zHRhRh//SE8LheR9G14/A+SybxfvlNnjSwPI/XmjvMjaC/+KWvQEy5Js5TR+FxvsAFy8UBFIFnC1zcfvQoGqaODvNEjFRhGGK+8Tlex/KeRyEm6PLxvHcG58WxBm/XtjO3QEyxwMd8UTFIT2e4mLyQxXsRF6aumQyK/1ekpSyJQt/tK/O2IfJCph1MFJkR87beVdT3i/y+e3Gck82QnydS5nZLrDVRhOLyhEiO6ChJab5IxHAIr7VQrvADjrKmi/PE02imX0jw+mgJgnIue8ozJi0ea65m3C3a7iTwnkainx0P2+U6T/0RevQ4zsm4SMjzu3jf16/nSQ3NJiYeVEXSju/jGPOkObNImiEiemLvQVaWiZBERNNH+Do7JEzxiYiKRX5s7x58GYN8dr7oejTKT0Z87SuVMHkjXeVjfrGCz6qwK8ZhHNtVrfOx2eg0IKYpElVUw/aeHD94rVCMsbKy7g/nMRFsrbBvAA3DMAzDMPoM2wAahmEYhmH0GbYBNAzDMAzD6DPWVAPoOvx38FQSdR6RMHnOKlqQbJ7r8JqKYepQnms4YoTGsN1lrkkKXdQEtcSLsMfGNkCM1CCedi6+8ftbd93Drx2hhiMutDGtOsYU8lzbkFBeZO6J89QVU9WD0xVWrlRQT9NxuLZh5DT8e2CyxO9PL8I+LC/wdiTaqJXJrhNG2U3lzeqroLbI7+nd//dfIebIzHFWdpXx8/APuD5U0y35UoOkmKF+5bNfZ2WpqyIiesYzn8HK3QSazlY7vA8PHEYj4cXF3fw8bazP9Aw3Gz5wcBfEXPDM81j5l97yPyDme9/+Hiv7y2goXRNm1S1CvdGB+7j+8Rvfn4WYbIzPr3hCGfPCSDinaACnNvK5+5JXvBxiViKm6Pt6QndWb6FmaqnKx1O5i3PSFwbykWKU3xbmsU4HY3rCAFwzI84W+TriKfojLybqo3wdAJo77TxCY6eZqssqhlqMOLcn9eJEFIR8TkarOI9WH6nvJQcbH4lr+Yr3uK8dXAFf0SQvLnPtVyGDmrKq0CnL+0dEFApj4UYLnzGuy8dUpBjc59P8PPNLGPPgw3ytyaZxzeq05fjFtidSvO+f2HMEYsYyfE+QzyovYxjnuvLFQ6hJdmL8vs/PY52npvizKlA0th2hrWw2ahDji8+FSj/nCvxZ0FVcuRvdp25ov1rsG0DDMAzDMIw+wzaAhmEYhmEYfYZtAA3DMAzDMPoM2wAahmEYhmH0GWuaBCKNn5sdFEJ7qSwrhx6KXZs9Lpb0UJdNyQQ3R4zHMxCTyHBD2UIBY2bn5/m112GCx+j6zax8bG4RYs668GJWbsxPQ8z+3U/wmDoaVcY8fkuKRTS8dISQduYYGosePiSMoJPY9sIYPzY8qFyrzRMonCU8z0CZC4YnR9GgeF1pHSvvewwFukSKObRgYoyboW7btBFiIuKiWc9FEa1MpNHMYiMhyE2ksO0U5+NwcnICQq649nmsnM9g4lMxVWLlxx95BGL2CHPW8XXrIaYd8XZ5aazzI7v3sPJju/dCTGbTaaw8fbwEMaUSPzaimOlmcrytSzMo8F46tp+V5xfmIaYd8HvRU4TZMxU+d579fIxZiXoNTV+rVS6+byjJW42GWOuUSxeEeW0yvbJRtaMkeKRj/HPxBJ5HJmvE4rjUeyLBLAgVsTlo0lGkLnMatKQL2R9hgElgMqFCNXAXMYFSH9mumJIsIc+dTOFzKCmTdkKsczKJn1uJ0hCuc4UCfy6mFIPipSo3VU8rCZQ9kTDQVcy0Y3E+phJJHD/dgCdvzC1hkkPH5+cZzKMR9NQWnrzR62HSTLXGn4OHjuLzNTHC1xY3wvNkhfG7M1qCmHyaz8FGBdt18NBhVt562hTEdMU62w1wryMeQ1QTpvhEROvFMzedwjW008JEsLXCvgE0DMMwDMPoM2wDaBiGYRiG0WfYBtAwDMMwDKPPWFMN4NgI30/2FtE8tiVeCt5AyQ1FLtctxBQz5EKB6ygScfztvCXMGVOKDoa6/Nj37/0uhGw+nevVjh1F/ZrUvWSSiuZG6B3Tij6rUeeau1YLTYx98fLurKIlevYzt7JyKo/6Pt/j/Rz0mhDTEpItt4aal5EMN7M877QzIGa0NMLK908fghgaxkOSpXk+pi6++CKIefYVl7JyMonjJwbmtfi3UCgMdz3C8/S6vA9bykvcF4/yti61Ub+ytMDbdUDo/YiIjs9xE+Xc6DjEUJJrEp0E6oS6Yvx89d++AzEbtp7FylODqG1MuXyMZxQT7E6b614OVJ+AmKwYm0GEuqWZMj/P8DAatjfFy9fv+rfvs/IZ27jOSmNhcQmOyXvcBnNbom6X92lc0fLEU0K718I1QmpRHRfHHMljkWJiHvAx5sYwJi3MhjW9oRT4qTpBAZgsE5GjiSIFzSZff8IAryW1jJoRtGyHVh/UFyr1EyGpFM6lpKKfW4l6E9eIKOT3a2IMF8OE0Pw1FZPwbIbPJSeGc8nxeMPiCexnR+j7mi2Miaf5+MkN4fzqubxdfgzXvpR42UAYw7lTEybY27ag/tmf4ZuJmQY+O6t1vs5u27YVYo4d2cfKPV/pH7F1alQrEBOKMZVVtN+5DG9ro4HPYC+DLw5YK+wbQMMwDMMwjD7DNoCGYRiGYRh9hm0ADcMwDMMw+gzbABqGYRiGYfQZa5oEsn49FzQWHEwY2HeEC2Bn0fOVugEX1uZyKIRuNLnRcRBiNokn9rdL8yjwrtWFwLtXhRgv4sfyOTS8nJ2psPJRRYAaCrH22AgagjohF/aWK2gWnczy/ikVUSSaEGLyThfFwCQMUhsd/HugW+f3NB5izLb1o6w8OY5G0EeO8hu9NI9C6NwqkkCyQri+WMV+fvAHPNFgRDEEHRuVBqUoqC6XRd+30ewzJu7Xus2YmDE1wO/P8d1o3N2s8ySCkbFRiDl/iI87L4WJPU2RNDQxgSamM8ePs/LCIo758Uk+nxzFlLfeEYLuGArie8I8N6EkPiWFSL+7iEaw5PJxOLoOk0B6Hd6HSpVXpNfr4sGIj3nNWFj6ASfTKYiReQaOsvp6Yt6GShsCsY4ESrKEJxJFvASuoa40BF6FYbJ2Lc2wWSI9lLWkq1KJj2/NNLgjkm0CB68tkz60+gXSUNpXrhXIY8p5FEPrlchkMRkg8Pk60lHWo5gwh1byHsEAXPuOR0wlisVXTuzphNg/jkjOzBRx/tdq0rwa58W8SOyLxfB5VkrzdmRKuPblUnztGx0pQMxiVOHnyeCYHxnlz+VaFc2i5eNUyTOiYpGP53wB73t1mT9jFhdwjxK5KyevnSr2DaBhGIZhGEafYRtAwzAMwzCMPsM2gIZhGIZhGH3GmmoACwNcXNBSdF4Do0KjoOghFma5zqPdRV1OLME1AEoIhT3+Q31PeWFztcV/g88opsrtpjBnbqPuo9uTpsqaVobvt+tV7J9CISXKqHVotfjnFhZRJ5jLca2VZvLqCFlHIob6DOErTAlFS7RxG9djtdDLkr5+D9fl/WA3ij8vPRM/B/URepVOG9v+rXvvYeWoh/e9kOEN0/RGbaGniyl/L23YtI6Vd1xyOsRs3cB1gctHjkPMTJnr3hJp1M9uHeK6wPl51L3uOJ134lnnbIOYT/zDP7FyjFBM1GvwPut2UWsZSYPUFI4fT4jjNm3eCDFzR/byA4r5cVroXs88Ew1c203eH1MTIyICtY6SoSHU5bqif4IAtWDSLFbTprXbvA8dTzExdoQGUDFe7godnqcJBQWoDSMKheG2bni7soGz1D+FSn18n18rVPrQE5oyX9Hl9cQxX4oLicgVbdX0WVIWKD9DROSS1D/itbT7sxJJ5RnjOvxYS3mgJUNex3QS561DvH8SccVIXIy7QhE12+0qnyvdGM6dWJK3vaWsEZ7H66gsxdRt8X6ebqMObnAdX0d60/j8SIs5l8rjej1S5ELzhcWjeK2ieOZK0SQR1X3ekNMnUPsdiud9s4njudngxwaKmF+gPJrWDPsG0DAMwzAMo8+wDaBhGIZhGEafYRtAwzAMwzCMPsM2gIZhGIZhGH3GmiaBxFL8dKkCiicHcsJUtYVi13iai0urZaWaAT9POiUF30SBSBgIOpgwEM8Ic03FzNbzeEJFJ1KE2b2VTWilLjxSRLOBOBRXzFkpwQWxlXIFQlpdbiRaVIwzYyIxxFXa3iQufJ5bqENMuc5VqvUGCoa/dvceVp5VEkUuxUNYH5EAQ0pyyzXXXcvKYVcxCe9JUbqStCOE4V4Mx3NSJDHNVPCe1ir7WHmphWJyRyRQ7H7wIMQsfmuBlbds3g4xF2zjyRG9Fqqu02L8RIrKWBpKux6Ow1BkEbUUQXxMmOlunMIkkHadG8GeWUCz6O99/2FWPn5oL8S0GnxQRc0KK289fQw+I9GSrsJAZBFEOOY6Yr5VmzjmpJGvpwj0IdFA8RmOizHvK/0eivPIhA8iIhIJJ06kZEusIsEkFIudOpdIGlwra2iL96GvmCGH0ozZxTrLGmuJGpGISqcw6UoaY7tKNolmCr4S0qSfiCiT4euIlnDiicHgKUlEQSD60MfnaySuX6thfVrC/Fhem4goJZ73MhGSiKjX4vO/uYzrUSLG254fxEQISvBnU6+pJJwk+D1NKEkyUZzXWTNnTopkpNLgEMRE1QorOy62vV3ja0CrieMwJe67NDH/4cVOwdF+ldg3gIZhGIZhGH2GbQANwzAMwzD6DNsAGoZhGIZh9BlrqgGs18XpPHypcy7Lf7uPp/H37YxwHy4W8bfzerUtynMY0xRGp208Tz7BjV9Tyhu2/Q7XLcRiuG9OiEPxJOp7pMlrJofd74pDvqIFSaT5uQsl1EyVl7hWr6ZobgqDvO1NHzU3ew9yU85dDx+DmNFBrpsam0JdBbn8+sPSbHOVZHP8/hQVeUR+hOvgOh3UnaTE3z4JB/WPkXhxeTKDMWGb6zxqNXxxuJfhbR3dihqXVoYbQe85cABiyBF6VaU+x6a5senQcAliBof59XuKc3enw3WczQb2YUfo3HodNDaPpfhYGJtEPc3haT53Zw8fhJh2nddn/6MPQ8zgEDd5jQbQ4HYlHOVvYkeId7uKm227w9ejnqJfc4X2SmpwiYgioZ/rKmbIHWGq7Cg6OGn8runXXBET+jiZ5BHNFlquLJFyrUDo8CIH1yO59sU8XIuhflqFImngjO0CaaOis3JBtI119hXd20pkEzhvY6JntW9mUkInXK8r2mahW04k8VppoVtOJFH/mBYVaC2jrntsdIqV24pOsJTl546PKOus6NYe4fySz8F0Dp95sYwYL8rY8MXYHB7JQkwiFNpPTfst+iyKUGuZyfD9T1rWj4hI3K9WC7WN8sUPa4l9A2gYhmEYhtFn2AbQMAzDMAyjz7ANoGEYhmEYRp9hG0DDMAzDMIw+Y02TQI4d4uVOBcWluREuak6lUSxdFLkjg4OYUFFvcGHkcgWFkuXFuChDCHmhNChFMTCYcoYodpU7aU2Y7QnT0FagiMCF5jseYv/4TZ7gESgi0UBca7mOMd2AJ3iUqyhAPbSXx1QWMWGg2+D9MVYchZgzNk6ycvUUda3NGjdVphD7MO5wYe/sLAqY9z52mJVTMUxcSRQLrDw0WoKYyWEeown7B4s8GUHxpaV2q8LKo6OYJLNukp9negYTn/bs3s3K3e4GiOl2uGC5VsP+aTbnWbm6jMktMgkk6OJN9ZL8Xjz2yCDEdDt8jI+Ooqn75Lln8pgRjBka4eMuJa4dEZqYSzTT4I6oX6+Hgu9ulwvXZR8TEXWF4bZmhuwI5boU9RMRpYQA3Y0phtIieSRS1jXZVsdVEtdkcoIyvjVjY0m7zfvHV5JbPHFure2yHVqCV0uYBGvmukmRUOEp67Xf5ffQdbCdqRQmNaxEXEs4EYbpCcV4fTX3Qt7ThJbUKJKIwhDX9JToj2IekyVkl6USuIaGXZm8gTG+mF9t5XkmE58yCeyfuDC4bzbxPMk8X1fbXRyHLTF34xFeyxNzxfVwryMf780WzvdKpcLK2rxIKElDa4V9A2gYhmEYhtFn2AbQMAzDMAyjz7ANoGEYhmEYRp+xphrAIM5NXnuJ8yCmGwpdhY/CvFSRiwtKIymIGXD5b+WDyouWB5b455YXUFPSavAuCHzFrDGShql4rXaLa1G03+09YSBda6MWpFUXRtmKwWTe5SLJ0EV91lyPtyuZxWul4ryOpQTqDTcTNw3e8Qw04Dz93B2svGnbZoi58BKuxzh6HE1MV0PY5f3jKn/DxHrCKDuO9+v+b9/LyjOzOA6dONd1XHjRMyDmsmefx8pVxTD14fu/z8qNNuqWdh/mBtsHDh6GmJbQtGjvCE8VuBlytYq6t5oQwzbEi82J0EM1prx4vpjnep7JzRshpjQ0xsqjk2MQM/nMs1h5oKCYs3or68OkUbact7XuyhrAXg81OFLzp+l05M2IxZSlVeiGNA9j2S5N5yWaRT0fNcny+qBjJiJH2Dx7ivGyvL6mp5O6vEjRUcr1UGtXu83nttbPcaFp08aBPLfWdqmRTChavkySj2/tfmn9sRLpBPazrGOk6Mzl/SkUUCcMuk6lflJ3FoXYz0Vhgp9TNHdRKEyMO8oYE47bYW8JYvJZodVV1jV55kYX19CYeOa1WoqhtHAbX1jGNaG+yJ+npRKa1y82Kqycks7ZRBQJ7WB5CbWWdbGmp9K410mnFdH4GmHfABqGYRiGYfQZtgE0DMMwDMPoM2wDaBiGYRiG0WfYBtAwDMMwDKPPcCLNIVTh1ltv/Q+uimEYhmEYhvHvYbX7NfsG0DAMwzAMo8+wDaBhGIZhGEafYRtAwzAMwzCMPsM2gIZhGIZhGH2GbQANwzAMwzD6DNsAGoZhGIZh9Bm2ATQMwzAMw+gzbANoGIZhGIbRZ6zaCNowDMMwDMP4r4F9A2gYhmEYhtFn2AbQMAzDMAyjz7ANoGEYhmEYRp9hG0DDMAzDMIw+wzaAhmEYhmEYfYZtAA3DMAzDMPoM2wAahmEYhmH0GbYBNAzDMAzD6DNsA2gYhmEYhtFn/P83JoHkFg5HAQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "NUM_IMAGES = 4\n", "images = [train_dataset[idx][0] for idx in range(NUM_IMAGES)]\n", "orig_images = [Image.fromarray(train_dataset.data[idx]) for idx in range(NUM_IMAGES)]\n", "orig_images = [test_transform(img) for img in orig_images]\n", "\n", "img_grid = torchvision.utils.make_grid(torch.stack(images + orig_images, dim=0), nrow=4, normalize=True, pad_value=0.5)\n", "img_grid = img_grid.permute(1, 2, 0)\n", "\n", "plt.figure(figsize=(8, 8))\n", "plt.title(\"Augmentation examples on CIFAR10\")\n", "plt.imshow(img_grid)\n", "plt.axis(\"off\")\n", "plt.show()\n", "plt.close()" ] }, { "cell_type": "markdown", "id": "a5550244-5d9c-4a41-ac48-91a2cf84e080", "metadata": {}, "source": [ "### Constructing a `LightningModule` " ] }, { "cell_type": "code", "execution_count": 76, "id": "0cec228e-dcb5-4bca-b32b-64bd1ca862d0", "metadata": {}, "outputs": [], "source": [ "class CIFARModule(L.LightningModule):\n", " def __init__(self, model, model_hparams, optimizer_hparams):\n", " \"\"\"\n", " model: PyTorch model that you plan to train\n", " model_hparams: Model hyperparameters, e.g., dropout, activation functions, etc.\n", " Not used in this example.\n", " optimizer_hparams: Optimizer hyperparameters, e.g., learning rate, etc.\n", " \"\"\"\n", " super().__init__()\n", "\n", " # Exports the hyperparameters to a YAML file, and create \"self.hparams\" namespace\n", " self.save_hyperparameters() \n", " self.model = model\n", " for param in self.model.parameters():\n", " param.requires_grad = False\n", " for param in self.model.get_classifier().parameters():\n", " param.requires_grad = True\n", " \n", " self.loss_module = nn.CrossEntropyLoss()\n", " # Example input for visualizing the graph in Tensorboard\n", " self.example_input_array = torch.zeros((1, 3, 32, 32), dtype=torch.float32) \n", "\n", " def forward(self, imgs):\n", " return self.model(imgs)\n", "\n", " def configure_optimizers(self):\n", " #\n", " # IMPORTANT\n", " #\n", " # Note that we are only passing classifiers parameters to the optimizer,\n", " # since we do not need to update the weights of the model backbone\n", " #\n", " optimizer = optim.AdamW(self.model.get_classifier().parameters(), **self.hparams.optimizer_hparams)\n", "\n", " # We will reduce the learning rate by 0.1 after 100 and 150 epochs\n", " scheduler = optim.lr_scheduler.MultiStepLR(optimizer, milestones=[100, 150], gamma=0.1)\n", " return [optimizer], [scheduler]\n", "\n", " def training_step(self, batch, batch_idx):\n", " # \"batch\" is the output of the training data loader.\n", " imgs, labels = batch\n", " preds = self.model(imgs)\n", " loss = self.loss_module(preds, labels)\n", " acc = (preds.argmax(dim=-1) == labels).float().mean()\n", "\n", " # Logs the accuracy per epoch to tensorboard (weighted average over batches)\n", " self.log(\"train_acc\", acc, on_step=False, on_epoch=True)\n", " self.log(\"train_loss\", loss)\n", " return loss # Return tensor to call \".backward\" on\n", "\n", " def validation_step(self, batch, batch_idx):\n", " imgs, labels = batch\n", " preds = self.model(imgs).argmax(dim=-1)\n", " acc = (labels == preds).float().mean()\n", " # By default logs it per epoch (weighted average over batches)\n", " self.log(\"val_acc\", acc)\n", "\n", " def test_step(self, batch, batch_idx):\n", " imgs, labels = batch\n", " preds = self.model(imgs).argmax(dim=-1)\n", " acc = (labels == preds).float().mean()\n", " # By default logs it per epoch (weighted average over batches), and returns it afterwards\n", " self.log(\"test_acc\", acc)" ] }, { "cell_type": "markdown", "id": "b3862a4d-1c57-4cfb-9fcf-fdc936d030aa", "metadata": {}, "source": [ "### Training method\n", "\n", "This methods includes the training, validation, test logic." ] }, { "cell_type": "code", "execution_count": 77, "id": "8727119e-3378-4f05-93d2-3f799fe93de7", "metadata": {}, "outputs": [], "source": [ "def train_model(model, save_name='resnet', **kwargs):\n", "\n", " # Create a PyTorch Lightning trainer with the generation callback\n", " trainer = L.Trainer(\n", " default_root_dir=os.path.join(CHECKPOINT_PATH, save_name), # Where to save models\n", " # We run on a single GPU (if possible)\n", " accelerator=\"auto\",\n", " devices=1,\n", " # How many epochs to train for if no patience is set\n", " max_epochs=2,\n", " callbacks=[\n", " ModelCheckpoint(\n", " save_weights_only=True, mode=\"max\", monitor=\"val_acc\"\n", " ), # Save the best checkpoint based on the maximum val_acc recorded. Saves only weights and not optimizer\n", " LearningRateMonitor(\"epoch\"),\n", " ], # Log learning rate every epoch\n", " ) # In case your notebook crashes due to the progress bar, consider increasing the refresh rate\n", " trainer.logger._log_graph = True # If True, we plot the computation graph in tensorboard\n", " trainer.logger._default_hp_metric = None # Optional logging argument that we don't need\n", "\n", " # Check whether pretrained model exists. If yes, load it and skip training\n", " pretrained_filename = os.path.join(CHECKPOINT_PATH, save_name + \".ckpt\")\n", " if os.path.isfile(pretrained_filename):\n", " print(f\"Found pretrained model at {pretrained_filename}, loading...\")\n", " # Automatically loads the model with the saved hyperparameters\n", " lightning_model = CIFARModule.load_from_checkpoint(pretrained_filename)\n", " else:\n", " print(\"No checkpoint found\")\n", " L.seed_everything(42) # To be reproducible\n", " lightning_model = CIFARModule(model, **kwargs)\n", " trainer.fit(lightning_model, train_loader, val_loader)\n", " lightning_model = CIFARModule.load_from_checkpoint(\n", " trainer.checkpoint_callback.best_model_path\n", " ) # Load best checkpoint after training\n", "\n", " # Test best model on validation and test set\n", " val_result = trainer.test(lightning_model, dataloaders=val_loader, verbose=False)\n", " test_result = trainer.test(lightning_model, dataloaders=test_loader, verbose=False)\n", " result = {\"test\": test_result[0][\"test_acc\"], \"val\": val_result[0][\"test_acc\"]}\n", "\n", " return model, result" ] }, { "cell_type": "markdown", "id": "f29062b5-6abc-4920-8989-ba0dcef510b1", "metadata": {}, "source": [ "### Now train" ] }, { "cell_type": "code", "execution_count": null, "id": "1442e09d-9d3e-40a3-988f-5c93fad79f5a", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "GPU available: True (mps), used: True\n", "TPU available: False, using: 0 TPU cores\n", "IPU available: False, using: 0 IPUs\n", "HPU available: False, using: 0 HPUs\n", "Seed set to 42\n", "\n", " | Name | Type | Params | In sizes | Out sizes\n", "------------------------------------------------------------------------------\n", "0 | model | ResNet | 21.3 M | [1, 3, 32, 32] | [1, 10] \n", "1 | loss_module | CrossEntropyLoss | 0 | ? | ? \n", "------------------------------------------------------------------------------\n", "5.1 K Trainable params\n", "21.3 M Non-trainable params\n", "21.3 M Total params\n", "85.159 Total estimated model params size (MB)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "No checkpoint found\n", "Epoch 0: 100%|███████████████████████| 351/351 [09:07<00:00, 0.64it/s, v_num=4]\n", "Validation: | | 0/? [00:00Complete the following tasks" ] }, { "cell_type": "markdown", "id": "d0a4222b-bc2f-4bd4-b1cc-257c9b1840bb", "metadata": {}, "source": [ "**1.** Modify the `train_model` function to take the number of epochs as an argument.\n", "\n", "**2.** Train the model for at least 20 epochs and plot the validation and test accuracies.\n", "\n", "Look at `saved_models/resnet/lightning_logs` for the this information.\n", " \n", "**3.** Modify the `train_model` function to load the model from *checkpoint*\n", "\n", "**4.** Write a function that uses the model trained in step 2 to perform inference in a list of images. The function prints the list of images alongside their true and predicted labels. We can use this function as follows:\n", "\n", "~~~python\n", "image_files = [\"1.png\", \"2.png\"]\n", "classify(image_files)\n", "~~~\n", "\n", "The output of this program will look similar to \n", "~~~text\n", "Classification results:\n", "1.png dog dog\n", "2.png cat dog\n", "~~~\n", " \n", "**5.** (Bonus) the current classification head consists of single fully-connected layer. Let's replace it with a neural network with one hidden layer. This can be achieve by replacing the \"fc\" layer with a *sequence* of layers as seen in the following code snippet.\n", "\n", "~~~python\n", "model.fc = nn.Sequential(\n", " nn.BatchNorm1d(num_in_features),\n", " nn.Linear(in_features=num_in_features, out_features=1024, bias=True),\n", " nn.ReLU(),\n", " nn.BatchNorm1d(1024),\n", " nn.Dropout(0.5)\n", " nn.Linear(in_features=1024, out_features=10, bias=True)\n", ")\n", "~~~\n", "\n", "where `num_in_features` simply reflect feature size for the ResNet encoder. You can find this information as follows\n", "\n", "~~~python\n", "num_in_features = resnet_cifar.get_classifier().in_features\n", "~~~\n", "\n" ] }, { "cell_type": "markdown", "id": "2414b369-30ec-4509-a0c3-61ed717d6e6b", "metadata": {}, "source": [ "
\n", "

Jupyter notebook

\n", "\n", "Source notebook is available here.\n", "
" ] }, { "cell_type": "markdown", "id": "520e8547-be60-4895-96f6-3ed2ba29298a", "metadata": {}, "source": [ "## GPU resources" ] }, { "cell_type": "markdown", "id": "b91e8a3d-e930-46de-9c79-f505b83c9875", "metadata": {}, "source": [ "(Experimental) You can use the GPU resources available at [https://hubdev.science.ontariotechu.ca/](https://hubdev.science.ontariotechu.ca/) to complete your lab. " ] }, { "cell_type": "markdown", "id": "c96b18cb-77f8-4f90-b2be-2195355e3a78", "metadata": {}, "source": [ "
\n", " \n", " \n", " \n", "
" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.7" } }, "nbformat": 4, "nbformat_minor": 5 }