Prerequisites

• Before starting the implementation of Task 1B, go through the following:
  • RL Controller Implementation
  • RL Learn Section
    

Problem Statement

• In this task, you will implement the Q Learning algorithm in order to make the robot follow the line provided in the given arena.
• The goal is to add your logic for state representation, rewards, action selection, and Q-learning updates for implementing a line following algorithm.

Procedure

• For this task we have provided a “Boilerplate” of code in Connector.py, task1b.py

  • Connector.py: Establish Connection with Bridge file (Do not edit it)
  • task1b.py: Q-learning logic (skeleton to be filled by you)

• Teams need to modify the following functions of file namely task1b.py.

  get_state() - how to turn the 5 sensor values into a discrete state.
get_reward() - how good the latest reading is.
choose_action() - which action to take in a given state (the policy).

Note: Reinforcement Learning models require repeated training iterations to learn optimal behavior, as they gradually improve through continuous interaction with the environment and refinement based on rewards and feedback.You can use extra function in connector.py to automate the training process.

• Once you built the Logic in task1b.py, open CoppeliaSim and load the provided task1b_scene.ttt scene file

• To train and run your solution, follow the steps given below:

1. Run the bridge application:

• Locate the bridge and task1b.py in File Explorer.Right Cleck and select open in Terminal Option.
• For Windows:

./bridge.exe

• For Ubuntu:

./bridge

2. Run your task1b.py program in new Terminal Window:

python task1b.py --mode train 

After Traning is complete/Program is closed you will have the q_table.pkl

Expected Output:

You can compare the execution of your python file with Bot in Coppeliasim: (Your implementation may look different, as the main aim is to follow line using RL)

WARNINGS !!

• Do not edit the name of any files / folders.
• Do not move/delete any files / folders.
• Do not edit the name, input or output arguments of any function.
• Teams are not allowed to edit any functions, other than the functions they are asked to develop.
• You are allowed to create any helper functions as you need in task1b.py file.
• While completing the function in task1b.py file, you might require to print some information on Terminal for the debugging process, but make sure to comment / remove all of them before submitting the scripts to us.
• You are not allowed to use any external Python Modules.