Merge pull request #45 from athul-2003/main

Added src/WikiSyllabus/KTU/2019_Scheme/S6/CSE/CST306/4_1dynamic_programming.md

README.md

@@ -22,6 +22,7 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
     <td align="center"><a href="https://github.com/IRJ2"><img src="https://avatars.githubusercontent.com/u/80707395?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Irene Rose Johnson</b></sub></a><br /><a href="https://github.com/gtechatfg/WikiSyllabus/commits?author=IRJ2" title="Documentation">📖</a></td>
     <td align="center"><a href="https://aravinds-arv.github.io/"><img src="https://avatars.githubusercontent.com/u/78845005?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Aravind S</b></sub></a><br /><a href="https://github.com/gtechatfg/WikiSyllabus/commits?author=aravinds-arv" title="Documentation">📖</a></td>
     <td align="center"><a href="https://github.com/milandeepak"><img src="https://avatars.githubusercontent.com/u/71485068?v=4?s=100" width="100px;" alt=""/><br /><sub><b>milandeepak</b></sub></a><br /><a href="#question-milandeepak" title="Answering Questions">💬</a> <a href="https://github.com/gtechatfg/WikiSyllabus/commits?author=milandeepak" title="Documentation">📖</a></td>
+    <td align="center"><a href="https://github.com/athul-2003"><img src="https://avatars.githubusercontent.com/u/128019369?s=400&u=51428c2c08488fb516f81735e347846f1fd34b34&v=4" width="100px;" alt=""/><br /><sub><b>H Athulkrishnan</b></sub></a><br /><a href="https://github.com/athul-2003/WikiSyllabus/commit/4ffba6a76a986a705aec1f07243ff0e923de4bae" title="Documentation">📖</a></td>
   </tr>
 </table>
 

src/WikiSyllabus/KTU/2019_Scheme/S6/CSE/CST306/4_1dynamic_programming.md

@@ -0,0 +1,133 @@
+# Dynamic Programming
+
+- Dynamic programming is an algorithm design method that can be used when the solution to a problem can be viewed as the result of a sequence of decisions.
+- Dynamic Programming is mainly an optimization over plain recursion.
+- Wherever we see a recursive solution that has repeated calls for same inputs, we can optimize it using Dynamic Programming.
+- The idea is to simply store the results of subproblems, so that we do not have to re-compute them when needed later.
+- This simple optimization reduces time complexities from exponential to polynomial.
+- For example, if we write simple recursive solution for Fibonacci Numbers, we get exponential time complexity and if we optimize it by storing solutions of subproblems, time complexity reduces to linear.
+- In dynamic programming an optimal sequence of decisions is obtained by making explicit appeal to the principle of optimality.
+
+## The Optimality Principle
+
+- **Definition**: The principle of optimality states that an optimal sequence of decisions has the property that whatever the initial state and decision are, the remaining decisions must constitute an optimal decision sequence with regard to the state resulting from the first decision.
+- A problem is said to satisfy the Principle of Optimality if the subsolutions of an optimal solution of the problem are themselves optimal solutions for their subproblems.
+- **Examples**:
+  - The shortest path problem satisfies the Principle of Optimality.
+  - This is because if a,x1,x2,...,xn,b is a shortest path from node a to node b in a graph, then the portion of xi to xj on that path is a shortest path from xi to xj.
+
+## Characteristics of Dynamic Programming
+
+1. **Overlapping Subproblems**
+   - Subproblems are smaller versions of the original problem. Any problem has overlapping sub-problems if finding its solution involves solving the same subproblem multiple times.
+   - Dynamic Programming also combines solutions to sub-problems. It is mainly used where the solution of one sub-problem is needed repeatedly. The computed solutions are stored in a table, so that these don’t have to be re-computed. Hence, this technique is needed where overlapping sub-problem exists.
+   - For example, Binary Search does not have overlapping sub-problem. Whereas recursive program of Fibonacci numbers have many overlapping sub-problems.
+
+2. **Optimal Substructure**
+   - A given problem has Optimal Substructure Property, if the optimal solution of the given problem can be obtained using optimal solutions of its sub-problems.
+   - For example, the Shortest Path problem has the following optimal substructure property: If a node x lies in the shortest path from a source node u to destination node v, then the shortest path from u to v is the combination of the shortest path from u to x, and the shortest path from x to v.
+
+## Steps of Dynamic Programming
+
+- Dynamic programming design involves 4 major steps:
+  1. Characterize the structure of an optimal solution.
+  2. Recursively define the value of an optimal solution.
+  3. Compute the value of an optimal solution, typically in a bottom-up fashion.
+  4. Construct an optimal solution from computed information.
+
+## Optimal Matrix Multiplication
+
+- Suppose we wish to compute the product of 4 matrices A1 x A2 x A3 x A4
+- The different parenthesizations are:
+  - ( A1(A2( A3 A4)))
+  - ( A1((A2 A3) A4))
+  - ( (A1A2)( A3 A4))
+  - (( A1(A2 A3))A4)
+  - ((( A1A2) A3)A4)
+- We can multiply two matrices A and B if and only if they are compatible: The number of columns of A must be equal to the number of rows of B.
+- If A is a pXq matrix and B is a qXr matrix, the resulting matrix C is a pX r matrix.
+- The time to compute C is pqr.
+- We shall express costs in terms of the number of scalar multiplications.
+- **Example**:
+  - Consider 3 matrices A1, A2 and A3. Its dimensions are 10X100, 100X5, 5X50 respectively.
+  - Number of scalar multiplications for:
+    - ((A1A2) A3) is 7500
+    - (A1 (A2A3)) is 75000
+  - Thus, computing the product according to the first parenthesization is 10 times faster.
+
+### Matrix-Chain Multiplication Problem
+
+- Given a chain (A1, A2, . . . An ) of n matrices, where for i = 1, 2, . . . n, matrix Ai has dimension pi-1 X pi , fully parenthesize the product A1A2. . . An in a way that minimizes the number of scalar multiplications.
+- In the matrix-chain multiplication problem, we are not actually multiplying matrices.
+- Our goal is only to determine an order for multiplying matrices that has the lowest cost.
+
+### Matrix Chain Multiplication: Dynamic Programming Method
+
+1. **The structure of an optimal parenthesization**
+   - Ai. .j denote the matrix that results from evaluating the product AiAi+1 . . . Aj where i <= j.
+   - If i < j, we must split the problem into two subproblems (Ai Ai+1 . . . Ak and Ak+1Ai+1 . . . Aj ), for some integer k in the range i <= k < j.
+   - That is, for some value of k, we first compute the matrices Ai. .k and Ak+1. .j. Then multiply them together to produce the final product Ai. .j .
+   - Total cost = Cost of computing the matrix Ai. .k + Cost of computing Ak+1. .j + Cost of multiplying them together.
+
+2. **A recursive solution**
+   - We can define the cost of an optimal solution recursively in terms of the optimal solutions to subproblems.
+   - Let m[i, j] be the minimum number of scalar multiplications needed to compute the matrix Ai. .j
+   - For the full problem, the lowest cost way to compute A1. .n would thus be m[1, n].
+   - Ai. .i = Ai so m[i,i] = 0 for i=1,2,. .. . n
+   - s[i,j] be a value of k at which we split the product AiAi+1 . . . Aj in an optimal parenthesization.
+   - This will take exponential time.
+
+3. **Computing the optimal costs**
+   - Compute the optimal cost by using a tabular, bottom-up approach.
+
+```python
+Algorithm Matrix_Chain_Order(p)
+{
+  n = p.length – 1
+  Let m[1..n, 1..n] and s[1..n-1 , 2..n] be new tables
+  for i=1 to n do
+    m[i,i] = 0
+  for l=2 to n do
+  {
+    for i=1 to n-l+1 do
+    {
+      j=i+l-1
+      m[i,j] = ∞
+      for k=i to j-1 do
+      {
+        q = m[i,k] + m[k+1,j] + pi-1 * pk * pj
+        if q < m[i,j] then
+        {
+          m[i,j] = q
+          s[i,j] = k
+        }
+      }
+    }
+  }
+  return m[][] and s[][]
+}
+```
+
+4. **Constructing an optimal solution**
+
+```python
+Algorithm Print_Optimal_Parens(s,i,j)
+{
+if i==j then
+print “A”i
+else
+print “(“
+print_Optimal_Parens(s,i,s[i,j])
+print_Optimal_Parens(s,s[i,j]+1,j)
+print “)”
+
+}
+```
+
+## Time Complexity
+- We are generating n(n-1)/2 number of elements in matrix m[].
+- To calculate each element it will take atmost n time.
+- So the time complexity = O( n.n(n-1)/2) = O(n3)
+
+
+

src/WikiSyllabus/KTU/2019_Scheme/S6/CSE/CST306/4_2floyd_warshall_algorithm.md

@@ -0,0 +1,33 @@
+# Floyd Warshall Algorithm
+
+- The Floyd Warshall Algorithm is for solving the All Pairs Shortest Path problem.
+- Negative edge weights are also allowed.
+- As a result of this algorithm, it will generate a matrix, which will represent the minimum
+distance from any node to all other nodes in the graph.
+- Inputs are the adjacency matrix of the given graph and the total number of vertices.
+
+```plaintext
+Algorithm FloydWarshall(cost[][], n)
+{
+    for i = 1 to n do
+        for j = 1 to n do
+            D[i, j] = cost[i, j]
+
+    for k := 1 to n do
+        for i := 1 to n do
+            for j := 1 to n do
+                D[i, j] = min{D[i, j], D[i, k] + D[k, j]}
+
+    Return D
+}
+```
+
+## Time Complexity
+
+- Floyd Warshall Algorithm consists of three loops over all the nodes. Each loop has constant complexities.
+- Hence, the time complexity of Floyd Warshall algorithm = O(n^3), where n is the number of nodes in the given graph.
+
+
+![Screenshot 2024-05-17 212206](https://github.com/athul-2003/WikiSyllabus/assets/128019369/1c065497-7227-45d6-8fbc-c05a9e86c2df)
+![Screenshot 2024-05-17 213114](https://github.com/athul-2003/WikiSyllabus/assets/128019369/45162c03-f3ef-4c7b-b7d8-d3757682791a)
+![Screenshot 2024-05-17 213245](https://github.com/athul-2003/WikiSyllabus/assets/128019369/090be10f-0569-4c84-9abe-996d4f852a60)

src/WikiSyllabus/KTU/2019_Scheme/S6/CSE/CST306/4_3backtracking.md

@@ -0,0 +1,45 @@
+# Backtracking
+
+- Backtracking method expresses the solution as an n-tuple (\(x_1, x_2, ..., x_n\)), where \(x_i\)'s are chosen from some finite set \(S_i\).
+- The problem to be solved calls for finding one vector that maximizes (or minimizes or satisfies) a criterion function \(P(x_1, x_2, ..., x_n)\).
+- Examples:
+  - Sorting the array of integers in \(a[l:n]\):
+    - The solution to the problem is expressed as an n-tuple, where \(x_i\) is the index of the ith smallest element.
+    - The criterion function \(P\) is: \(a[x_i] \leq a[x_{i+1}]\), for \(1 \leq i < n\).
+    - The set \(S_i = \{1, 2, ..., n\}\).
+  - Different methods for solving this problem:
+    - **Brute Force approach**:
+      - Suppose \(m_i\) is the size of set \(S_i\).
+      - The number of tuples (with size n) that are possible candidates for satisfying the function \(P\) is: \(m = m_1 \times m_2 \times m_3 \times ... \times m_n\).
+      - Brute Force approach evaluates each one with \(P\), and saves those which yield the optimum.
+
+    - **Backtracking algorithm**:
+      - It yields the same answer with far fewer than \(m\) trials.
+      - Its basic idea is to build up the solution vector one component at a time and to use modified criterion functions (bounding functions) \(P_i(x_1, x_2, ..., x_i)\) to test whether the vector being formed has any chance of success.
+      - The major advantage of this method is that if it is realized that the partial vector (\(x_1, x_2, ..., x_i\)) can in no way lead to an optimal solution, then \(m_{i+1} \times m_{i+2} \times ... \times m_n\) possible test vectors can be ignored entirely.
+
+- Backtracking method requires that all the solutions satisfy a complex set of constraints. These constraints can be divided into two categories:
+    - **Explicit Constraints**:
+      - Explicit constraints are rules that restrict each \(x_i\) to take on values only from a given set.
+      - The explicit constraints depend on the particular instance \(I\) of the problem being solved. All tuples that satisfy the explicit constraints define a possible solution space for \(I\).
+      - Example:
+        ![Screenshot 2024-05-17 215544](https://github.com/athul-2003/WikiSyllabus/assets/128019369/d1d95e2d-1359-4d98-853d-19de2926e044)
+
+        
+    - **Implicit Constraints**:
+      - These are rules that determine which of the tuples in the solution space of \(I\) satisfy the criterion function (Bounding Function).
+
+# Backtracking Control Abstraction
+
+- \( (x_1, x_2, ..., x_i) \) be a path from the root to a node in a state space tree.
+- Generating Function \( T(x_1, x_2, ..., x_i) \) be the set of all possible values for \( x_{i+1} \) such that \( (x_1, x_2, ..., x_{i+1}) \) is also a path to a problem state.
+- \( T(x_1, x_2, ..., x_n) = \emptyset \)
+- Bounding function \( B_{i+1}(x_1, x_2, ..., x_{i+1}) \) is false for a path \( (x_1, x_2, ..., x_{i+1}) \) from the root node to a problem state, then the path cannot be extended to reach an answer node.
+- Thus the candidates for position \( i+1 \) of the solution vector \( (x_1, x_2, ..., x_n) \) are those values which are generated by \( T \) and satisfy \( B_{i+1} \).
+- The recursive version is initially invoked by Backtrack(1).
+    ![Screenshot 2024-05-17 220015](https://github.com/athul-2003/WikiSyllabus/assets/128019369/27df2798-aad6-4ddf-a945-30af1bb2d66a)
+
+- All the possible elements for the kth position of the tuple that satisfy \( B_k \) are generated one by one, and adjoined to the current vector \( (x_1, x_2, ..., x_{k-1}) \).
+- Each time \( x_k \) is attached, a check is made to determine whether a solution has been found. Then the algorithm is recursively invoked.
+- When the for loop is exited, no more values for \( x_k \) exist and the current copy of Backtrack ends. The last unresolved call now resumes.
+- This algorithm causes all solutions to be printed. If only a single solution is desired, then a flag can be added as a parameter to indicate the first occurrence of success.

src/WikiSyllabus/KTU/2019_Scheme/S6/CSE/CST306/4_4n_queens_problem().md

@@ -0,0 +1,70 @@
+# N-Queens Problem
+
+![Screenshot 2024-05-17 222513](https://github.com/athul-2003/WikiSyllabus/assets/128019369/e3d29214-7ae4-49f2-b186-f59a570b4e13)
+
+- For example, the following is a solution for the 4 Queen problem.
+
+![Screenshot 2024-05-17 222900](https://github.com/athul-2003/WikiSyllabus/assets/128019369/8b8955e5-cbf0-459e-b94b-5369c1f8f6d2)
+
+
+- Consider an \(n * n\) chessboard and try to find all possible ways to place \(n\) non-attacking queens.
+- \( (x_1, x_2, ..., x_n) \) be the solution vector. Queen \(i\) is placed in the \(i^{th}\) row and \(x_i^{th}\) column. \(x_i\) will all be distinct since no two queens can be placed in the same column.
+
+- There are 2 types of diagonals:
+  - **Positive Diagonal**:
+    - Diagonal from upper left to lower right.
+    - Every element on the same diagonal has the same row-column value.
+    - Suppose 2 queens are placed at position \((i,j)\) and \((k,l)\), then \(i-j = k-l\).
+  - **Negative Diagonal**:
+    - Diagonal from upper right to lower left.
+    - Every element on the same diagonal has the same row+column value.
+    - Suppose 2 queens are placed at position \((i,j)\) and \((k,l)\), then \(i+j = k+l\).
+    - The 1st equation implies: \(i-k = j-l\).
+    - The 2nd equation implies: \(i-k = l-j\).
+    - Combining these two, we will get: \(|i-k| = |j-l|\). Absolute value of column difference is equal to the absolute value of row difference.
+
+```cpp
+Algorithm NQueens(k,n)
+{
+    for i=1 to n do
+    { 
+        if Place(k,i) then
+        { 
+            x[k] = i
+            if(k==n) then write(x[1:n])
+            else NQueens(k+1, n)
+        }
+    }
+}
+```
+```cpp
+Algorithm Place(k,i)
+{
+    for j=1 to k-1 do
+    {
+        if( (x[j]==i) or ( Abs(j-k)==Abs(x[j]-i) ) then
+            Return false
+    }
+    Return true
+}
+```
+
+- `Place(k,i)` returns true if the \(k^{th}\) queen can be placed in column \(i\).
+  - \(i\) should be distinct from all previous values \(x[1]\), \(x[2]\), ..., \(x[k-1]\).
+  - And no 2 queens are to be on the same diagonal.
+  - Time complexity of Place() is \(O(k)\).
+
+- `NQueen()` is initially invoked by `NQueen(1,n)`.
+  - Time complexity of NQueen() is \(O(n!)\).
+
+# Backtracking works on 4-Queens Problem
+
+- If \( (x_1, x_2, ..., x_i) \) is the path to the current E-node, then all children nodes with parent-child labeling \( x_{i+1} \) are such that \( (x_1, x_2, ..., x_{i+1}) \) represents a chessboard configuration in which no two queens are attacking.
+- We start with the root node as the only live node. This becomes the E-node and the path is ().
+- We generate one child (node 2) and the path is now (1). This corresponds to placing queen 1 on column 1.
+- Node 2 becomes the E-node. Node 3 is generated and immediately killed.
+- The next node generated is node 8 and the path becomes (1,3). Node 8 becomes the E-node. However, it gets killed as all its children represent board configurations that cannot lead to an answer node.
+- We backtrack to node 2 and generate another child node 13. The path is now (1,4). This process continues until it will generate a proper arrangement.
+
+
+![Screenshot 2024-05-17 223544](https://github.com/athul-2003/WikiSyllabus/assets/128019369/ae28eebb-42ed-405a-8159-e29c729cb4d3)

src/WikiSyllabus/KTU/2019_Scheme/S6/CSE/CST306/SUMMARY.md

@@ -0,0 +1 @@
+CST306