Thread by @rezakakooee

Why has reinforcement learning not been adapted to the design process?

I believe the RL problem and the design process are pretty similar, and the RL community should embrace the design process as a potential application for RL algorithms. (1/N)

In this thread, I will shortly elaborate on why these two are similar in my eyes.

Design is a crucial step in making things, but it is not easy to find a single definition for it. In the context of architectural design, ... (2/N)

... one might define the design process as a series of steps followed by the designer to iteratively find a solution for a given design scenario.

In his book, Notes on the Synthesis of Form, in 1964, Christopher Alexander wrote:

"The ultimate object of design is form". (3/N)

He further explained that form does not exist in a vacuum but lives and changes a broader context.

"The form is the solution to the problem; the context defines the problem".

The designer aims at creating a form that satisfies various requirements of its context. (4/N)

The designer perceives the context and starts the design process by giving birth to an initial form that alters the context.

In an iterative process, the designer adapts the form to reach the design scenario's objective. (5/N)

I think you have already started to perceive the similarity between the design process and RL. If not, let's discuss what the RL problem is about.

In RL, the learner (agent) learns how to achieve its goal by interacting with the surrounding environment. (6/N)

To compare the RL problem with the design process, we consider the environment as the context and the RL agent as the artificial designer who aims to achieve a goal that is comparable to making the form. (7/N)

Environment/context is the entity that determines the boundaries of the learning/design process.

Like the design process, RL is centered around the idea of trial-and-error learning in order to optimize a notion of objective function. (8/N)

The goal of the RL agent is to learn a behavioral policy that knows which action to choose in every state of the environment that is analogous to the series of steps the human designer takes to construct a form.

The behavioral policy the agent learns by interacting with...(9/N)

... the environment resembles the expertise a human designer learns throughout her academic/professional journey.

Now you might see why I believe the RL problem and the design process are similar, and why RL researchers/practitioners need to see the design as an ...(10/N)

... exciting and potential application of RL.

We recently wrote and submitted a paper showing how RL can be used in space layout design. I will write more about it when the paper is accepted. If you are interested make me happy by seeing me here @RezaKakooee again :) N=11.

#Design #RL #MachineLearning #ArtificialIntelligence #ReinforcementLearning