Category: Greedy Algorithms

  • Optimal Merge Pattern Algorithm

    Merge a set of sorted files of different length into a single sorted file. We need to find an optimal solution, where the resultant file will be generated in minimum time.

    If the number of sorted files are given, there are many ways to merge them into a single sorted file. This merge can be performed pair wise. Hence, this type of merging is called as 2-way merge patterns.

    As, different pairings require different amounts of time, in this strategy we want to determine an optimal way of merging many files together. At each step, two shortest sequences are merged.

    To merge a p-record file and a q-record file requires possibly p + q record moves, the obvious choice being, merge the two smallest files together at each step.

    Two-way merge patterns can be represented by binary merge trees. Let us consider a set of n sorted files {f1, f2, f3, , fn}. Initially, each element of this is considered as a single node binary tree. To find this optimal solution, the following algorithm is used.

    Pseudocode

    Following is the pseudocode of the Optimal Merge Pattern Algorithm −

     for i := 1 to n  1 do  
       declare new node  
       node.leftchild := least (list) 
       node.rightchild := least (list) 
       node.weight) := ((node.leftchild).weight)+
       ((node.rightchild).weight)  
       insert (list, node);  
    return least (list); 
    

    At the end of this algorithm, the weight of the root node represents the optimal cost.

    Examples

    Let us consider the given files, f1, f2, f3, f4 and f5 with 20, 30, 10, 5 and 30 number of elements respectively.

    If merge operations are performed according to the provided sequence, then

    M1 = merge f1 and f2 => 20 + 30 = 50

    M2 = merge M1 and f3 => 50 + 10 = 60

    M3 = merge M2 and f4 => 60 + 5 = 65

    M4 = merge M3 and f5 => 65 + 30 = 95

    Hence, the total number of operations is

    50 + 60 + 65 + 95 = 270

    Now, the question arises is there any better solution?

    Sorting the numbers according to their size in an ascending order, we get the following sequence −

    f4, f3, f1, f2, f5

    Hence, merge operations can be performed on this sequence

    M1 = merge f4 and f3 => 5 + 10 = 15

    M2 = merge M1 and f1 => 15 + 20 = 35

    M3 = merge M2 and f2 => 35 + 30 = 65

    M4 = merge M3 and f5 => 65 + 30 = 95

    Therefore, the total number of operations is

    15 + 35 + 65 + 95 = 210

    Obviously, this is better than the previous one.

    In this context, we are now going to solve the problem using this algorithm.

    Initial Set

    Initial Set

    Step 1

    Step-1

    Step 2

    Initial Set

    Step 3

    Initial Set

    Step 4

    Initial Set

    Hence, the solution takes 15 + 35 + 60 + 95 = 205 number of comparisons.

    Example

    Following are the implementations of the above approach in various programming languages −

    CC++JavaPython

    #include <stdio.h>#include <stdlib.h>intoptimalMerge(int files[],int n){// Sort the files in ascending orderfor(int i =0; i < n -1; i++){for(int j =0; j < n - i -1; j++){if(files[j]> files[j +1]){int temp = files[j];
                    files[j]= files[j +1];
                    files[j +1]= temp;}}}int cost =0;while(n >1){// Merge the smallest two filesint mergedFileSize = files[0]+ files[1];
            cost += mergedFileSize;// Replace the first file with the merged file size
            files[0]= mergedFileSize;// Shift the remaining files to the leftfor(int i =1; i < n -1; i++){
                files[i]= files[i +1];}
            n--;// Reduce the number of files// Sort the files againfor(int i =0; i < n -1; i++){for(int j =0; j < n - i -1; j++){if(files[j]> files[j +1]){int temp = files[j];
                        files[j]= files[j +1];
                        files[j +1]= temp;}}}}return cost;}intmain(){int files[]={5,10,20,30,30};int n =sizeof(files)/sizeof(files[0]);int minCost =optimalMerge(files, n);printf("Minimum cost of merging is: %d Comparisons\n", minCost);return0;}

    Output

    Minimum cost of merging is: 205 Comparisons
  • Job Sequencing with Deadline

    Job scheduling algorithm is applied to schedule the jobs on a single processor to maximize the profits.

    The greedy approach of the job scheduling algorithm states that, Given n number of jobs with a starting time and ending time, they need to be scheduled in such a way that maximum profit is received within the maximum deadline.

    Job Scheduling Algorithm

    Set of jobs with deadlines and profits are taken as an input with the job scheduling algorithm and scheduled subset of jobs with maximum profit are obtained as the final output.

    Algorithm

    Step1 −  Find the maximum deadline value from the input set 
    of jobs.
    Step2 −  Once, the deadline is decided, arrange the jobs 
    in descending order of their profits.
    Step3 −  For each job in the sorted order, find the latest 
    available time slot that is less than or equal to its deadline 
    and assign the job to that slot.
    Step4 −  The selected set of jobs are the output.
    

    Examples

    Consider the following tasks with their deadlines and profits. Schedule the tasks in such a way that they produce maximum profit after being executed −

    S. No.12345
    JobsJ1J2J3J4J5
    Deadlines22134
    Profits20604010080

    Step 1

    Find the maximum deadline value, dm, from the deadlines given.

    dm = 4
    

    This means we have 4 time slots available: [1], [2], [3], [4]

    Step 2

    Arrange the jobs in descending order of their profits.

    S. No.12345
    JobsJ4J5J2J3J1
    Deadlines34212
    Profits10080604020

    Step 3

    Select jobs greedily, placing each in the latest available time slot that does not exceed its deadline.

    Iteration 1: J4 (Profit = 100, Deadline = 3)

    Find the latest free slot ≤ 3. Slot 3 is free, so assign J4 to slot 3.

    Slots: [_, _, J4, _]
    Total Profit = 100
    

    Iteration 2: J5 (Profit = 80, Deadline = 4)

    Find the latest free slot ≤ 4. Slot 4 is free, so assign J5 to slot 4.

    Slots: [_, _, J4, J5]
    Total Profit = 100 + 80 = 180
    

    Iteration 3: J2 (Profit = 60, Deadline = 2)

    Find the latest free slot ≤ 2. Slot 2 is free, so assign J2 to slot 2.

    Slots: [_, J2, J4, J5]
    Total Profit = 180 + 60 = 240
    

    Iteration 4: J3 (Profit = 40, Deadline = 1)

    Find the latest free slot ≤ 1. Slot 1 is free, so assign J3 to slot 1.

    Slots: [J3, J2, J4, J5]
    Total Profit = 240 + 40 = 280
    

    Iteration 5: J1 (Profit = 20, Deadline = 2)

    Find the latest free slot ≤ 2. Slots 1 and 2 are already occupied. J1 cannot be scheduled.

    Step 4

    All jobs have been considered. The algorithm terminates.

    Final Result:

    The optimal sequence of jobs scheduled within their deadlines is {J3, J2, J4, J5} (executed in time slots 1, 2, 3, 4 respectively) with the maximum profit of 280.

    Time Slot1234
    JobJ3J2J4J5
    Profit406010080
    Maximum Profit: 40 + 60 + 100 + 80 = 280
    

    Example

    Following is the final implementation of Job sequencing Algorithm using Greedy Approach −

    CC++JavaPython

    #include <stdbool.h>#include <stdio.h>#include <stdlib.h>// A structure to represent a JobtypedefstructJob{char id[3];// Job Id (e.g., "J1", "J2")int deadline;// Deadline of Jobint profit;// Profit if Job is completed before or on deadline} Job;// Comparison function for sorting jobs by profit in descending orderintcompare(constvoid* a,constvoid* b){
       Job* job1 =(Job*)a;
       Job* job2 =(Job*)b;return(job2->profit - job1->profit);}// Find minimum between two numbersintmin(int num1,int num2){return(num1 < num2)? num1 : num2;}intmain(){// Jobs from the example: J1, J2, J3, J4, J5// Deadlines: 2, 2, 1, 3, 4// Profits: 20, 60, 40, 100, 80
       Job arr[]={{"J1",2,20},{"J2",2,60},{"J3",1,40},{"J4",3,100},{"J5",4,80}};int n =sizeof(arr)/sizeof(arr[0]);// Find maximum deadlineint maxDeadline =0;for(int i =0; i < n; i++){if(arr[i].deadline > maxDeadline)
             maxDeadline = arr[i].deadline;}printf("Maximum Deadline: %d\n", maxDeadline);printf("Number of time slots available: %d\n\n", maxDeadline);// Sort jobs by profit in descending orderqsort(arr, n,sizeof(Job), compare);printf("Jobs sorted by profit (descending):\n");for(int i =0; i < n; i++){printf("%s (Deadline: %d, Profit: %d)\n", arr[i].id, arr[i].deadline, arr[i].profit);}printf("\n");int result[maxDeadline];// To store result sequence of jobs
       bool slot[maxDeadline];// To keep track of free time slots// Initialize all slots to be freefor(int i =0; i < maxDeadline; i++)
          slot[i]= false;int totalProfit =0;// Iterate through all given jobsfor(int i =0; i < n; i++){// Find a free slot for this job (start from the last possible slot)for(int j =min(maxDeadline, arr[i].deadline)-1; j >=0; j--){// Free slot foundif(slot[j]== false){
                result[j]= i;
                slot[j]= true;
                totalProfit += arr[i].profit;printf("Scheduled %s in slot %d (Profit: %d)\n", arr[i].id, j +1, arr[i].profit);break;}}}// Print the resultprintf("\nFollowing is the maximum profit sequence of Jobs:\n");for(int i =0; i < maxDeadline; i++){if(slot[i])printf("Slot %d: %s\n", i +1, arr[result[i]].id);}printf("\nTotal Maximum Profit: %d\n", totalProfit);return0;}

    Output

    Maximum Deadline: 4
    Number of time slots available: 4
    
    Jobs sorted by profit (descending):
    J4 (Deadline: 3, Profit: 100)
    J5 (Deadline: 4, Profit: 80)
    J2 (Deadline: 2, Profit: 60)
    J3 (Deadline: 1, Profit: 40)
    J1 (Deadline: 2, Profit: 20)
    
    Scheduled J4 in slot 3 (Profit: 100)
    Scheduled J5 in slot 4 (Profit: 80)
    Scheduled J2 in slot 2 (Profit: 60)
    Scheduled J3 in slot 1 (Profit: 40)
    
    Following is the maximum profit sequence of Jobs:
    Slot 1: J3
    Slot 2: J2
    Slot 3: J4
    Slot 4: J5
    
    Total Maximum Profit: 280
  • Fractional Knapsack Problem

    The knapsack problem states that − given a set of items, holding weights and profit values, one must determine the subset of the items to be added in a knapsack such that, the total weight of the items must not exceed the limit of the knapsack and its total profit value is maximum.

    It is one of the most popular problems that take greedy approach to be solved. It is called as the Fractional Knapsack Problem.

    To explain this problem a little easier, consider a test with 12 questions, 10 marks each, out of which only 10 should be attempted to get the maximum mark of 100. The test taker now must calculate the highest profitable questions the one that hes confident in to achieve the maximum mark. However, he cannot attempt all the 12 questions since there will not be any extra marks awarded for those attempted answers. This is the most basic real-world application of the knapsack problem.

    Knapsack Algorithm

    The weights (Wi) and profit values (Pi) of the items to be added in the knapsack are taken as an input for the fractional knapsack algorithm and the subset of the items added in the knapsack without exceeding the limit and with maximum profit is achieved as the output.

    Algorithm

    • Consider all the items with their weights and profits mentioned respectively.
    • Calculate Pi/Wi of all the items and sort the items in descending order based on their Pi/Wi values.
    • Without exceeding the limit, add the items into the knapsack.
    • If the knapsack can still store some weight, but the weights of other items exceed the limit, the fractional part of the next time can be added.
    • Hence, giving it the name fractional knapsack problem.

    Examples

    • For the given set of items and the knapsack capacity of 10 kg, find the subset of the items to be added in the knapsack such that the profit is maximum.
    Items12345
    Weights (in kg)33251
    Profits101510128

    Solution

    Step 1

    Given, n = 5

    Wi = {3, 3, 2, 5, 1}
    Pi = {10, 15, 10, 12, 8}
    

    Calculate Pi/Wi for all the items

    Items12345
    Weights (in kg)33251
    Profits101510208
    Pi/Wi3.35548

    Step 2

    Arrange all the items in descending order based on Pi/Wi

    Items52341
    Weights (in kg)13253
    Profits815102010
    Pi/Wi85543.3

    Step 3

    Without exceeding the knapsack capacity, insert the items in the knapsack with maximum profit.

    Knapsack = {5, 2, 3}
    

    However, the knapsack can still hold 4 kg weight, but the next item having 5 kg weight will exceed the capacity. Therefore, only 4 kg weight of the 5 kg will be added in the knapsack.

    Items52341
    Weights (in kg)13253
    Profits815102010
    Knapsack1114/50

    Hence, the knapsack holds the weights = [(1 * 1) + (1 * 3) + (1 * 2) + (4/5 * 5)] = 10, with maximum profit of [(1 * 8) + (1 * 15) + (1 * 10) + (4/5 * 20)] = 37.

    Example

    Following is the final implementation of Fractional Knapsack Algorithm using Greedy Approach −

    CC++JavaPython

    #include <stdio.h>int n =5;int p[10]={3,3,2,5,1};int w[10]={10,15,10,12,8};int W =10;intmain(){int cur_w;float tot_v;int i, maxi;int used[10];for(i =0; i < n;++i)
          used[i]=0;
       cur_w = W;while(cur_w >0){
          maxi =-1;for(i =0; i < n;++i)if((used[i]==0)&&((maxi ==-1)||((float)w[i]/p[i]>(float)w[maxi]/p[maxi])))
                maxi = i;
          used[maxi]=1;
          cur_w -= p[maxi];
          tot_v += w[maxi];if(cur_w >=0)printf("Added object %d (%d, %d) completely in the bag. Space left: %d.\n", maxi +1, w[maxi], p[maxi], cur_w);else{printf("Added %d%% (%d, %d) of object %d in the bag.\n",(int)((1+(float)cur_w/p[maxi])*100), w[maxi], p[maxi], maxi +1);
             tot_v -= w[maxi];
             tot_v +=(1+(float)cur_w/p[maxi])* w[maxi];}}printf("Filled the bag with objects worth %.2f.\n", tot_v);return0;}

    Output

    Added object 5 (8, 1) completely in the bag. Space left: 9.
    Added object 2 (15, 3) completely in the bag. Space left: 6.
    Added object 3 (10, 2) completely in the bag. Space left: 4.
    Added object 1 (10, 3) completely in the bag. Space left: 1.
    Added 19% (12, 5) of object 4 in the bag.
    Filled the bag with objects worth 45.40.
    

    Applications

    Few of the many real-world applications of the knapsack problem are −

    • Cutting raw materials without losing too much material
    • Picking through the investments and portfolios
    • Selecting assets of asset-backed securitization
    • Generating keys for the Merkle-Hellman algorithm
    • Cognitive Radio Networks
    • Power Allocation
    • Network selection for mobile nodes
    • Cooperative wireless communication
  • Map Colouring Algorithm

    Map colouring problem states that given a graph G {V, E} where V and E are the set of vertices and edges of the graph, all vertices in V need to be coloured in such a way that no two adjacent vertices must have the same colour.

    The real-world applications of this algorithm are assigning mobile radio frequencies, making schedules, designing Sudoku, allocating registers etc.

    Map Colouring Algorithm

    With the map colouring algorithm, a graph G and the colours to be added to the graph are taken as an input and a coloured graph with no two adjacent vertices having the same colour is achieved.

    Algorithm

    • Initiate all the vertices in the graph.
    • Select the node with the highest degree to colour it with any colour.
    • Choose the colour to be used on the graph with the help of the selection colour function so that no adjacent vertex is having the same colour.
    • Check if the colour can be added and if it does, add it to the solution set.
    • Repeat the process from step 2 until the output set is ready.

    Examples

    Map_Colouring_graph

    Step 1

    Find degrees of all the vertices −

    A  4
    B  2
    C  2
    D  3
    E  3
    

    Step 2

    Choose the vertex with the highest degree to colour first, i.e., A and choose a colour using selection colour function. Check if the colour can be added to the vertex and if yes, add it to the solution set.

    highest_degree

    Step 3

    Select any vertex with the next highest degree from the remaining vertices and colour it using selection colour function.

    D and E both have the next highest degree 3, so choose any one between them, say D.

    d_highest_degree

    D is adjacent to A, therefore it cannot be coloured in the same colour as A. Hence, choose a different colour using selection colour function.

    Step 4

    The next highest degree vertex is E, hence choose E.

    E_highest_degree

    E is adjacent to both A and D, therefore it cannot be coloured in the same colours as A and D. Choose a different colour using selection colour function.

    Step 5

    The next highest degree vertices are B and C. Thus, choose any one randomly.

    B_and_C_highest_degree

    B is adjacent to both A and E, thus not allowing to be coloured in the colours of A and E but it is not adjacent to D, so it can be coloured with Ds colour.

    Step 6

    The next and the last vertex remaining is C, which is adjacent to both A and D, not allowing it to be coloured using the colours of A and D. But it is not adjacent to E, so it can be coloured in Es colour.

    C_highest_degree

    Example

    Following is the complete implementation of Map Colouring Algorithm in various programming languages where a graph is coloured in such a way that no two adjacent vertices have same colour.

    CC++JavaPython

    #include<stdio.h>#include<stdbool.h>#define V 4
    bool graph[V][V]={{0,1,1,0},{1,0,1,1},{1,1,0,1},{0,1,1,0},};
    bool isValid(int v,int color[],int c){//check whether putting a color valid for vfor(int i =0; i < V; i++)if(graph[v][i]&& c == color[i])return false;return true;}
    bool mColoring(int colors,int color[],int vertex){if(vertex == V)//when all vertices are consideredreturn true;for(int col =1; col <= colors; col++){if(isValid(vertex,color, col)){//check whether color col is valid or not
             color[vertex]= col;if(mColoring(colors, color, vertex+1)== true)//go for additional verticesreturn true;
             color[vertex]=0;}}return false;//when no colors can be assigned}intmain(){int colors =3;// Number of colorsint color[V];//make color matrix for each vertexfor(int i =0; i < V; i++)
          color[i]=0;//initially set to 0if(mColoring(colors, color,0)== false){//for vertex 0 check graph coloringprintf("Solution does not exist.");}printf("Assigned Colors are: \n");for(int i =0; i < V; i++)printf("%d ", color[i]);return0;}

    Output

    Assigned Colors are:
    1 2 3 1
  • Dijkstras Shortest Path Algorithm

    Dijkstras shortest path algorithm is similar to that of Prims algorithm as they both rely on finding the shortest path locally to achieve the global solution. However, unlike prims algorithm, the dijkstras algorithm does not find the minimum spanning tree; it is designed to find the shortest path in the graph from one vertex to other remaining vertices in the graph. Dijkstras algorithm can be performed on both directed and undirected graphs.

    Since the shortest path can be calculated from single source vertex to all the other vertices in the graph, Dijkstras algorithm is also called single-source shortest path algorithm. The output obtained is called shortest path spanning tree.

    In this chapter, we will learn about the greedy approach of the dijkstras algorithm.

    Dijkstras Algorithm

    The dijkstras algorithm is designed to find the shortest path between two vertices of a graph. These two vertices could either be adjacent or the farthest points in the graph. The algorithm starts from the source. The inputs taken by the algorithm are the graph G {V, E}, where V is the set of vertices and E is the set of edges, and the source vertex S. And the output is the shortest path spanning tree.

    Algorithm

    • Declare two arrays − distance[] to store the distances from the source vertex to the other vertices in graph and visited[] to store the visited vertices.
    • Set distance[S] to 0 and distance[v] = ∞, where v represents all the other vertices in the graph.
    • Add S to the visited[] array and find the adjacent vertices of S with the minimum distance.
    • The adjacent vertex to S, say A, has the minimum distance and is not in the visited array yet. A is picked and added to the visited array and the distance of A is changed from ∞ to the assigned distance of A, say d1, where d1 < ∞.
    • Repeat the process for the adjacent vertices of the visited vertices until the shortest path spanning tree is formed.

    Examples

    To understand the dijkstras concept better, let us analyze the algorithm with the help of an example graph −

    Dijkstras graph

    Step 1

    Initialize the distances of all the vertices as ∞, except the source node S.

    VertexSABCDE
    Distance0

    Now that the source vertex S is visited, add it into the visited array.

    visited = {S}
    

    Step 2

    The vertex S has three adjacent vertices with various distances and the vertex with minimum distance among them all is A. Hence, A is visited and the dist[A] is changed from ∞ to 6.

    S → A = 6
    S → D = 8
    S → E = 7
    
    VertexSABCDE
    Distance0687
    Visited = {S, A}
    
    Visited s to a

    Step 3

    There are two vertices visited in the visited array, therefore, the adjacent vertices must be checked for both the visited vertices.

    Vertex S has two more adjacent vertices to be visited yet: D and E. Vertex A has one adjacent vertex B.

    Calculate the distances from S to D, E, B and select the minimum distance −

    S → D = 8 and S → E = 7.
    S → B = S → A + A → B = 6 + 9 = 15
    
    VertexSABCDE
    Distance061587
    Visited = {S, A, E}
    
    Visited_S_A_E

    Step 4

    Calculate the distances of the adjacent vertices S, A, E of all the visited arrays and select the vertex with minimum distance.

    S → D = 8
    S → B = 15
    S → C = S → E + E → C = 7 + 5 = 12
    
    VertexSABCDE
    Distance06151287
    Visited = {S, A, E, D}
    
    Visited_s_a_e_d

    Step 5

    Recalculate the distances of unvisited vertices and if the distances minimum than existing distance is found, replace the value in the distance array.

    S → C = S → E + E → C = 7 + 5 = 12
    S → C = S → D + D → C = 8 + 3 = 11
    

    dist[C] = minimum (12, 11) = 11

    S → B = S → A + A → B = 6 + 9 = 15
    S → B = S → D + D → C + C → B = 8 + 3 + 12 = 23
    

    dist[B] = minimum (15,23) = 15

    VertexSABCDE
    Distance06151187
    Visited = { S, A, E, D, C}
    
    Visited_S_A_E_D_C

    Step 6

    The remaining unvisited vertex in the graph is B with the minimum distance 15, is added to the output spanning tree.

    Visited = {S, A, E, D, C, B}
    
    Visited_S_A_E_D_C_B

    The shortest path spanning tree is obtained as an output using the dijkstras algorithm.

    Example

    The program implements the dijkstras shortest path problem that takes the cost adjacency matrix as the input and prints the shortest path as the output along with the minimum cost.

    CC++JavaPython

    #include<stdio.h>#include<limits.h>#include<stdbool.h>intmin_dist(int[], bool[]);voidgreedy_dijsktra(int[][6],int);intmin_dist(int dist[], bool visited[]){// finding minimum distint minimum=INT_MAX,ind;for(int k=0; k<6; k++){if(visited[k]==false && dist[k]<=minimum){
             minimum=dist[k];
             ind=k;}}return ind;}voidgreedy_dijsktra(int graph[6][6],int src){int dist[6];
       bool visited[6];for(int k =0; k<6; k++){
          dist[k]= INT_MAX;
          visited[k]= false;}
       dist[src]=0;// Source vertex dist is set 0for(int k =0; k<6; k++){int m=min_dist(dist,visited);
          visited[m]=true;for(int k =0; k<6; k++){// updating the dist of neighbouring vertexif(!visited[k]&& graph[m][k]&& dist[m]!=INT_MAX && dist[m]+graph[m][k]<dist[k])
                dist[k]=dist[m]+graph[m][k];}}printf("Vertex\t\tdist from source vertex\n");for(int k =0; k<6; k++){char str=65+k;printf("%c\t\t\t%d\n", str, dist[k]);}}intmain(){int graph[6][6]={{0,1,2,0,0,0},{1,0,0,5,1,0},{2,0,0,2,3,0},{0,5,2,0,2,2},{0,1,3,2,0,1},{0,0,0,2,1,0}};greedy_dijsktra(graph,0);return0;}

    Output

    Vertex		dist from source vertex
    A			   0
    B			   1
    C			   2
    D			   4
    E			   2
    F			   3
  • Kruskals Minimal Spanning Tree Algorithm

    Kruskal’s minimal spanning tree algorithm is one of the efficient methods to find the minimum spanning tree of a graph. A minimum spanning tree is a subgraph that connects all the vertices present in the main graph with the least possible edges and minimum cost (sum of the weights assigned to each edge).

    The algorithm first starts from the forest which is defined as a subgraph containing only vertices of the main graph of the graph, adding the least cost edges later until the minimum spanning tree is created without forming cycles in the graph.

    Kruskal’s algorithm has easier implementation than prims algorithm, but has higher complexity.

    Kruskal’s Algorithm

    The inputs taken by the kruskals algorithm are the graph G {V, E}, where V is the set of vertices and E is the set of edges, and the source vertex S and the minimum spanning tree of graph G is obtained as an output.

    Algorithm

    • Sort all the edges in the graph in an ascending order and store it in an array edge[].
    • Construct the forest of the graph on a plane with all the vertices in it.
    • Select the least cost edge from the edge[] array and add it into the forest of the graph. Mark the vertices visited by adding them into the visited[] array.
    • Repeat the steps 2 and 3 until all the vertices are visited without having any cycles forming in the graph
    • When all the vertices are visited, the minimum spanning tree is formed.
    • Calculate the minimum cost of the output spanning tree formed.

    Examples

    Construct a minimum spanning tree using kruskals algorithm for the graph given below −

    kruskals_algorithm_graph

    Solution

    As the first step, sort all the edges in the given graph in an ascending order and store the values in an array.

    EdgeB→DA→BC→FF→EB→CG→FA→GC→DD→EC→G
    Cost56910111215172225

    Then, construct a forest of the given graph on a single plane.

    graph_on_single_plane

    From the list of sorted edge costs, select the least cost edge and add it onto the forest in output graph.

    B → D = 5
    Minimum cost = 5
    Visited array, v = {B, D}
    
    sorted_edge_costs

    Similarly, the next least cost edge is B → A = 6; so we add it onto the output graph.

    Minimum cost = 5 + 6 = 11
    Visited array, v = {B, D, A}
    
    graph_b_to_a

    The next least cost edge is C → F = 9; add it onto the output graph.

    Minimum Cost = 5 + 6 + 9 = 20
    Visited array, v = {B, D, A, C, F}
    
    graph_c_to_f

    The next edge to be added onto the output graph is F → E = 10.

    Minimum Cost = 5 + 6 + 9 + 10 = 30
    Visited array, v = {B, D, A, C, F, E}
    
    output_graph_e_to_f

    The next edge from the least cost array is B → C = 11, hence we add it in the output graph.

    Minimum cost = 5 + 6 + 9 + 10 + 11 = 41
    Visited array, v = {B, D, A, C, F, E}
    
    least_cost_array

    The last edge from the least cost array to be added in the output graph is F → G = 12.

    Minimum cost = 5 + 6 + 9 + 10 + 11 + 12 = 53
    Visited array, v = {B, D, A, C, F, E, G}
    
    output_graph_f_to_g

    The obtained result is the minimum spanning tree of the given graph with cost = 53.

    Example

    The final program implements the Kruskals minimum spanning tree problem that takes the cost adjacency matrix as the input and prints the shortest path as the output along with the minimum cost.

    CC++JavaPython

    #include <stdio.h>#include <stdlib.h>constint inf =999999;int k, a, b, u, v, n, ne =1;int mincost =0;int cost[3][3]={{0,10,20},{12,0,15},{16,18,0}};int  p[9]={0};intapplyfind(int i){while(p[i]!=0)
            i=p[i];return i;}intapplyunion(int i,int j){if(i!=j){
            p[j]=i;return1;}return0;}intmain(){
        n =3;int i, j;for(int i =0; i < n; i++){for(int j =0; j < n; j++){if(cost[i][j]==0){
                    cost[i][j]= inf;}}}printf("Minimum Cost Spanning Tree: \n");while(ne < n){int min_val = inf;for(i=0; i<n; i++){for(j=0; j <n; j++){if(cost[i][j]< min_val){
                        min_val = cost[i][j];
                        a = u = i;
                        b = v = j;}}}
            u =applyfind(u);
            v =applyfind(v);if(applyunion(u, v)!=0){printf("%d -> %d\n", a, b);
                mincost +=min_val;}
            cost[a][b]= cost[b][a]=999;
            ne++;}printf("Minimum cost = %d",mincost);return0;}

    Output

    Minimum Cost Spanning Tree: 
    0 -> 1
    1 -> 2
    Minimum cost = 25
  • Prims Minimal Spanning Tree

    Prim’s minimal spanning tree algorithm is one of the efficient methods to find the minimum spanning tree of a graph. A minimum spanning tree is a sub graph that connects all the vertices present in the main graph with the least possible edges and minimum cost (sum of the weights assigned to each edge).

    The algorithm, similar to any shortest path algorithm, begins from a vertex that is set as a root and walks through all the vertices in the graph by determining the least cost adjacent edges.

    spanning_tree_of_graph

    Prim’s Algorithm

    To execute the prim’s algorithm, the inputs taken by the algorithm are the graph G {V, E}, where V is the set of vertices and E is the set of edges, and the source vertex S. A minimum spanning tree of graph G is obtained as an output.

    Algorithm

    • Declare an array visited[] to store the visited vertices and firstly, add the arbitrary root, say S, to the visited array.
    • Check whether the adjacent vertices of the last visited vertex are present in the visited[] array or not.
    • If the vertices are not in the visited[] array, compare the cost of edges and add the least cost edge to the output spanning tree.
    • The adjacent unvisited vertex with the least cost edge is added into the visited[] array and the least cost edge is added to the minimum spanning tree output.
    • Steps 2 and 4 are repeated for all the unvisited vertices in the graph to obtain the full minimum spanning tree output for the given graph.
    • Calculate the cost of the minimum spanning tree obtained.

    Examples

    • Find the minimum spanning tree using prims method (greedy approach) for the graph given below with S as the arbitrary root.
    minimum spanning tree

    Solution

    Step 1

    Create a visited array to store all the visited vertices into it.

    V = { }
    

    The arbitrary root is mentioned to be S, so among all the edges that are connected to S we need to find the least cost edge.

    S → B = 8
    V = {S, B}
    
    s to b

    Step 2

    Since B is the last visited, check for the least cost edge that is connected to the vertex B.

    B → A = 9
    B → C = 16
    B → E = 14
    

    Hence, B → A is the edge added to the spanning tree.

    V = {S, B, A}
    
    b to a

    Step 3

    Since A is the last visited, check for the least cost edge that is connected to the vertex A.

    A → C = 22
    A → B = 9
    A → E = 11
    

    But A → B is already in the spanning tree, check for the next least cost edge. Hence, A → E is added to the spanning tree.

    V = {S, B, A, E}
    
    a to e

    Step 4

    Since E is the last visited, check for the least cost edge that is connected to the vertex E.

    E → C = 18
    E → D = 3
    

    Therefore, E → D is added to the spanning tree.

    V = {S, B, A, E, D}
    
    e to d

    Step 5

    Since D is the last visited, check for the least cost edge that is connected to the vertex D.

    D → C = 15
    E → D = 3
    

    Therefore, D → C is added to the spanning tree.

    V = {S, B, A, E, D, C}
    
    d to c

    The minimum spanning tree is obtained with the minimum cost = 46

    Example

    The final program implements Prims minimum spanning tree problem that takes the cost adjacency matrix as the input and prints the spanning tree as the output along with the minimum cost.

    CC++JavaPython

    #include<stdio.h>#include<stdlib.h>#define inf 99999#define MAX 10int G[MAX][MAX]={{0,19,8},{21,0,13},{15,18,0}};int S[MAX][MAX], n;intprims();intmain(){int i, j, cost;
       n =3;
       cost=prims();printf("Spanning tree:");for(i=0; i<n; i++){printf("\n");for(j=0; j<n; j++)printf("%d\t",S[i][j]);}printf("\nMinimum cost = %d", cost);return0;}intprims(){int C[MAX][MAX];int u, v, min_dist, dist[MAX], from[MAX];int visited[MAX],ne,i,min_cost,j;//create cost[][] matrix,spanning[][]for(i=0; i<n; i++)for(j=0; j<n; j++){if(G[i][j]==0)
                C[i][j]=inf;else
                C[i][j]=G[i][j];
             S[i][j]=0;}//initialise visited[],distance[] and from[]
       dist[0]=0;
       visited[0]=1;for(i=1; i<n; i++){
          dist[i]= C[0][i];
          from[i]=0;
          visited[i]=0;}
       min_cost =0;//cost of spanning tree
       ne = n-1;//no. of edges to be addedwhile(ne >0){//find the vertex at minimum distance from the tree
          min_dist = inf;for(i=1; i<n; i++)if(visited[i]==0&& dist[i]< min_dist){
                v = i;
                min_dist = dist[i];}
          u = from[v];//insert the edge in spanning tree
          S[u][v]= dist[v];
          S[v][u]= dist[v];
          ne--;
          visited[v]=1;//updated the distance[] arrayfor(i=1; i<n; i++)if(visited[i]==0&& C[i][v]< dist[i]){
                dist[i]= C[i][v];
                from[i]= v;}
          min_cost = min_cost + C[u][v];}return(min_cost);}

    Output

    Spanning tree:
    0	0	8	
    0	0	13	
    8	13	0	
    Minimum cost = 26
  • Travelling Salesman Problem (Greedy Approach)

    The travelling salesman problem is a graph computational problem where the salesman needs to visit all cities (represented using nodes in a graph) in a list just once and the distances (represented using edges in the graph) between all these cities are known. The solution that is needed to be found for this problem is the shortest possible route in which the salesman visits all the cities and returns to the origin city.

    If you look at the graph below, considering that the salesman starts from the vertex a, they need to travel through all the remaining vertices b, c, d, e, f and get back to a while making sure that the cost taken is minimum.

    salesman_graph

    There are various approaches to find the solution to the travelling salesman problem: naive approach, greedy approach, dynamic programming approach, etc. In this tutorial we will be learning about solving travelling salesman problem using greedy approach.

    Travelling Salesperson Algorithm

    As the definition for greedy approach states, we need to find the best optimal solution locally to figure out the global optimal solution. The inputs taken by the algorithm are the graph G {V, E}, where V is the set of vertices and E is the set of edges. The shortest path of graph G starting from one vertex returning to the same vertex is obtained as the output.

    Algorithm

    • Travelling salesman problem takes a graph G {V, E} as an input and declare another graph as the output (say G) which will record the path the salesman is going to take from one node to another.
    • The algorithm begins by sorting all the edges in the input graph G from the least distance to the largest distance.
    • The first edge selected is the edge with least distance, and one of the two vertices (say A and B) being the origin node (say A).
    • Then among the adjacent edges of the node other than the origin node (B), find the least cost edge and add it onto the output graph.
    • Continue the process with further nodes making sure there are no cycles in the output graph and the path reaches back to the origin node A.
    • However, if the origin is mentioned in the given problem, then the solution must always start from that node only. Let us look at some example problems to understand this better.

    Examples

    Consider the following graph with six cities and the distances between them −

    graph_six_cities

    From the given graph, since the origin is already mentioned, the solution must always start from that node. Among the edges leading from A, A → B has the shortest distance.

    graph a to b

    Then, B → C has the shortest and only edge between, therefore it is included in the output graph.

    graph_b_to_c

    Theres only one edge between C → D, therefore it is added to the output graph.

    graph_c_to_d

    Theres two outward edges from D. Even though, D → B has lower distance than D → E, B is already visited once and it would form a cycle if added to the output graph. Therefore, D → E is added into the output graph.

    graph d to e

    Theres only one edge from e, that is E → F. Therefore, it is added into the output graph.

    graph e to f

    Again, even though F → C has lower distance than F → A, F → A is added into the output graph in order to avoid the cycle that would form and C is already visited once.

    graph f to a

    The shortest path that originates and ends at A is A → B → C → D → E → F → A

    The cost of the path is: 16 + 21 + 12 + 15 + 16 + 34 = 114.

    Even though, the cost of path could be decreased if it originates from other nodes but the question is not raised with respect to that.

    Example

    The complete implementation of Travelling Salesman Problem using Greedy Approach is given below −

    CC++JavaPython

    #include <stdio.h>int tsp_g[10][10]={{12,30,33,10,45},{56,22,9,15,18},{29,13,8,5,12},{33,28,16,10,3},{1,4,30,24,20}};int visited[10], n, cost =0;/* creating a function to generate the shortest path */voidtravellingsalesman(int c){int k, adj_vertex =999;int min =999;/* marking the vertices visited in an assigned array */
       visited[c]=1;/* displaying the shortest path */printf("%d ", c +1);/* checking the minimum cost edge in the graph */for(k =0; k < n; k++){if((tsp_g[c][k]!=0)&&(visited[k]==0)){if(tsp_g[c][k]< min){
                min = tsp_g[c][k];
                adj_vertex = k;}}}if(min !=999){
          cost = cost + min;}if(adj_vertex ==999){
          adj_vertex =0;printf("%d", adj_vertex +1);
          cost = cost + tsp_g[c][adj_vertex];return;}travellingsalesman(adj_vertex);}/* main function */intmain(){int i, j;
       n =5;for(i =0; i < n; i++){
          visited[i]=0;}printf("Shortest Path: ");travellingsalesman(0);printf("\nMinimum Cost: ");printf("%d\n", cost);return0;}

    Output

    Shortest Path: 1 4 5 2 3 1
    Minimum Cost: 55
  • Greedy Algorithms

    Among all the algorithmic approaches, the simplest and straightforward approach is the Greedy method. In this approach, the decision is taken on the basis of current available information without worrying about the effect of the current decision in future.

    Greedy algorithms build a solution part by part, choosing the next part in such a way, that it gives an immediate benefit. This approach never reconsiders the choices taken previously. This approach is mainly used to solve optimization problems. Greedy method is easy to implement and quite efficient in most of the cases. Hence, we can say that Greedy algorithm is an algorithmic paradigm based on heuristic that follows local optimal choice at each step with the hope of finding global optimal solution.

    In many problems, it does not produce an optimal solution though it gives an approximate (near optimal) solution in a reasonable time.

    Components of Greedy Algorithm

    Greedy algorithms have the following five components −

    • A candidate set − A solution is created from this set.
    • A selection function − Used to choose the best candidate to be added to the solution.
    • A feasibility function − Used to determine whether a candidate can be used to contribute to the solution.
    • An objective function − Used to assign a value to a solution or a partial solution.
    • A solution function − Used to indicate whether a complete solution has been reached.

    Areas of Application

    Greedy approach is used to solve many problems, such as

    • Finding the shortest path between two vertices using Dijkstra’s algorithm.
    • Finding the minimal spanning tree in a graph using Prim’s /Kruskal’s algorithm, etc.

    Counting Coins Problem

    The Counting Coins problem is to count to a desired value by choosing the least possible coins and the greedy approach forces the algorithm to pick the largest possible coin. If we are provided coins of 1, 2, 5 and 10 and we are asked to count 18 then the greedy procedure will be −

    • 1 − Select one 10 coin, the remaining count is 8
    • 2 − Then select one 5 coin, the remaining count is 3
    • 3 − Then select one 2 coin, the remaining count is 1
    • 4 − And finally, the selection of one 1 coins solves the problem

    Though, it seems to be working fine, for this count we need to pick only 4 coins. But if we slightly change the problem then the same approach may not be able to produce the same optimum result.

    For the currency system, where we have coins of 1, 7, 10 value, counting coins for value 18 will be absolutely optimum but for count like 15, it may use more coins than necessary. For example, the greedy approach will use 10 + 1 + 1 + 1 + 1 + 1, total 6 coins. Whereas the same problem could be solved by using only 3 coins (7 + 7 + 1)

    Hence, we may conclude that the greedy approach picks an immediate optimized solution and may fail where global optimization is a major concern.

    Where Greedy Approach Fails

    In many problems, Greedy algorithm fails to find an optimal solution, moreover it may produce a worst solution. Problems like Travelling Salesman and Knapsack cannot be solved using this approach.

    Examples of Greedy Algorithm

    Most networking algorithms use the greedy approach. Here is a list of few of them −

    • Travelling Salesman Problem
    • Prim’s Minimal Spanning Tree Algorithm
    • Kruskal’s Minimal Spanning Tree Algorithm
    • Dijkstra’s Minimal Spanning Tree Algorithm
    • Graph − Map Coloring
    • Graph − Vertex Cover
    • DSA – Fractional Knapsack Problem
    • DSA – Job Sequencing with Deadline
    • DSA – Optimal Merge Pattern Algorithm

    We will discuss these examples elaborately in the further chapters of this tutorial.