done ex3

PW-3/ex3/ex3-review-questions-stud.ipynb

@@ -21,7 +21,7 @@
    "id": "8d2fb7ef",
    "metadata": {},
    "source": [
-    "TODO"
+    "For each class, we must compute the likelyhood, which is one calculus per class, so O(nb_class). Then, for each x we must compute the posteriori probability, which is looking into a pre-computed histogram (done in the training phase), so this is O(nb_class). The a priori probability only needs to be computed for each class, so O(nb_class). So, the total complexity of the Bayesian classifier is O(2 * nb_class * nb_x)"
    ]
   },
   {
@@ -38,7 +38,9 @@
    "id": "88ab64b2",
    "metadata": {},
    "source": [
-    "TODO"
+    "To generate a new sample, we need to create a y and a x. This can be done by firstly picking the class Ck randomly according to the a priori probabilities P(Ck). This means that, if there is two classes and P(C1) = 0.6 and P(C2) = 0.4, we take 60% of the time C1 and 40% of the time C2\n",
+    "\n",
+    "Then, we can pick a random x based on the probability density function p(x|Ck). This means we choose a class and, in the density function (like histogram), we take a random x based on the probablilities. If there is two x values possibles and one is distributed as 0.4 and the other 0.6, we will taxe x1 40% of the time and x2 60%"
    ]
   },
   {
@@ -51,12 +53,44 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "id": "14aba0f7",
    "metadata": {},
    "outputs": [],
    "source": [
-    "pass"
+    "def generateSample(priors, histValues, edgeValues, n):\n",
+    "    # pick a class according to the proba\n",
+    "    # to do that, compute the different probabilities sum. This is done by creating intervals between 0 and 1. The size of those intervals represents the probability of the random\n",
+    "    # number generator to land on it.\n",
+    "    cumulative_probs = np.cumsum(priors)\n",
+    "\n",
+    "    # take a random number and see in which interval it falls. The index of this interval will be the class we chose\n",
+    "    chosen_class = 0\n",
+    "    r = random.random() \n",
+    "    for i, cp in enumerate(cumulative_probs):\n",
+    "        if r < cp:\n",
+    "            chosen_class = i\n",
+    "            break\n",
+    "    \n",
+    "\n",
+    "    # The same logic is used to find the new x value. We take the proba of x given c and chose randomly weighted by those proba.\n",
+    "    # we have to compute the \"probabilities\" differently, because the histogram is only the count of each x in the c.\n",
+    "    # here, we kept the count instead of proba and when generating the random number, instead of chosing between 0 1 and 1 we chose between 0 and total_hist\n",
+    "    # which does the same job in the end\n",
+    "    total_hist = np.sum(histValues[chosen_class])\n",
+    "\n",
+    "    cumulative_probs_hist = np.cumsum(histValues[chosen_class])\n",
+    "\n",
+    "    # take a random number and see in which interval it falls. The index of this interval will be the class we chose\n",
+    "    chosen_x_index = 0\n",
+    "    r = random.uniform(0, total_hist) \n",
+    "    for i, cp in enumerate(cumulative_probs_hist):\n",
+    "        if r < cp:\n",
+    "            chosen_x_index = i\n",
+    "            break\n",
+    "\n",
+    "    chosen_x = edgeValues[chosen_x_index]\n",
+    "    \n"
    ]
   },
   {
@@ -72,7 +106,7 @@
    "id": "4bb03365",
    "metadata": {},
    "source": [
-    "TODO"
+    "If we only take the priors, then A has 5/6 chances to be chosen vs B 1/6. The minimum accuracy is then 5/6, because the system will choose based on the 5/6 and 1/6, which will yeild 5/6 good results"
    ]
   },
   {
@@ -90,7 +124,10 @@
    "id": "d2bf1500",
    "metadata": {},
    "source": [
-    "TODO"
+    "The a priori probability is simply the repartition of each class in the dataset.\n",
+    "The likelihood is the tricky part, because the system would need to be multivariate (because of all the pixels), which makes it very complex. We could use the Naive Bayes formula witch states that the features (pixels) are completely uncorrelated and then we cound perform the operation pixel per pixel for each image. However, the pixels ARE NOT uncorrelated, because a pixel is spatially positionned. If a pixel is white, there is a strong change that there are white pixels somewhere around as well. The Naive Bayes would technically still work-ish, but with a false presomption.\n",
+    "\n",
+    "To do it correctly, we would have to use something like the multivariate gaussian distribution"
    ]
   },
   {
@@ -128,7 +165,9 @@
    "id": "1adf1760",
    "metadata": {},
    "source": [
-    "TODO"
+    "1. The system can make false positives or false negatives. This means it could say that an innocent man is a threat or that a dangerous person is safe to cross the border.\n",
+    "2. Yes, a false negative is the most critical one. In the case of a false positive, the only consequence is a lost of time because you have to interrogate the \"suspect\", maybe resulting in a angry customer. On the other hand, a false negative means a real threat has entered the country and has not been detect, wich could have way more concequences than an angry customer.\n",
+    "3. In this case, we could use the Area Under the Curve with this system. This would allow to tune the the treshold and impact the decision to tend more to false positives rather than false negatives"
    ]
   },
   {
@@ -144,7 +183,9 @@
    "id": "fa5ffd45-0645-4093-9a1b-0a7aeaeece0e",
    "metadata": {},
    "source": [
-    "TODO"
+    "The bayes equation : P(Ck|x) = (p(x|Ck)*P(Ck))/p(x).\n",
+    "\n",
+    "The a priori probability (P(Ck)) is what reprensents the unbalance in the training set. This value is the probability to have this class, so it is linked to the number of data in it. This means that a class with 3x more data in it will have a way bigger P(Ck) witch will impact the decision in favor of the biggest P(Ck)."
    ]
   }
  ],