Reza Kakooee Profile picture
Jul 3 12 tweets 4 min read
Why has reinforcement learning not been adapted to the design process?

I believe the RL problem and the design process are pretty similar, and the RL community should embrace the design process as a potential application for RL algorithms. (1/N)
In this thread, I will shortly elaborate on why these two are similar in my eyes.

Design is a crucial step in making things, but it is not easy to find a single definition for it. In the context of architectural design, ... (2/N)
... one might define the design process as a series of steps followed by the designer to iteratively find a solution for a given design scenario.

In his book, Notes on the Synthesis of Form, in 1964, Christopher Alexander wrote:

"The ultimate object of design is form". (3/N)
He further explained that form does not exist in a vacuum but lives and changes a broader context.

"The form is the solution to the problem; the context defines the problem".

The designer aims at creating a form that satisfies various requirements of its context. (4/N)
The designer perceives the context and starts the design process by giving birth to an initial form that alters the context.

In an iterative process, the designer adapts the form to reach the design scenario's objective. (5/N)
I think you have already started to perceive the similarity between the design process and RL. If not, let's discuss what the RL problem is about.

In RL, the learner (agent) learns how to achieve its goal by interacting with the surrounding environment. (6/N)
To compare the RL problem with the design process, we consider the environment as the context and the RL agent as the artificial designer who aims to achieve a goal that is comparable to making the form. (7/N)
Environment/context is the entity that determines the boundaries of the learning/design process.

Like the design process, RL is centered around the idea of trial-and-error learning in order to optimize a notion of objective function. (8/N)
The goal of the RL agent is to learn a behavioral policy that knows which action to choose in every state of the environment that is analogous to the series of steps the human designer takes to construct a form.

The behavioral policy the agent learns by interacting with...(9/N)
... the environment resembles the expertise a human designer learns throughout her academic/professional journey.

Now you might see why I believe the RL problem and the design process are similar, and why RL researchers/practitioners need to see the design as an ...(10/N)
... exciting and potential application of RL.

We recently wrote and submitted a paper showing how RL can be used in space layout design. I will write more about it when the paper is accepted. If you are interested make me happy by seeing me here @RezaKakooee again :) N=11.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Reza Kakooee

Reza Kakooee Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(