Import Modules¶

We will need some basic packages, but will also need to import different AutoRA theorists. Here, we will import two theorists: Differentiable Architecture Search, and Logistic Regression.

Note that we are currently re-structuring the AutoRA package so that all experimentalists and theorists are their own sub-package. As such, once this is complete for the theorists, these imports should be modified to conform to the new structure.

In [ ]:

# Uncomment the following line when running on Google Colab
# !pip install autora
# !pip install "autora[theorist-darts]"

# Uncomment the following line when running on Google Colab # !pip install autora # !pip install "autora[theorist-darts]"

In [3]:

import numpy as np
import matplotlib.pyplot as plt
from autora.theorist.darts import DARTSRegressor; DARTSRegressor()
from autora.experimentalist.leverage import leverage_sample

import numpy as np import matplotlib.pyplot as plt from autora.theorist.darts import DARTSRegressor; DARTSRegressor() from autora.experimentalist.leverage import leverage_sample

Define Meta-Space¶

We will here define X values of interest as well as a ground truth model to derive y values.

In [4]:

#Define X with some noise
X = np.linspace(start=-3, stop=6, num=25).reshape(-1, 1)
noise = np.array([np.random.normal(0,.5) for r in range(len(X))]).reshape(-1,1)
X = X + noise
X.reshape(-1).sort(kind='mergesort')

#Define ground truth model
def ground_truth(xs):
    y = (xs ** 2.0)
    y[xs < 0] = 0
    return y

#Define X with some noise X = np.linspace(start=-3, stop=6, num=25).reshape(-1, 1) noise = np.array([np.random.normal(0,.5) for r in range(len(X))]).reshape(-1,1) X = X + noise X.reshape(-1).sort(kind='mergesort') #Define ground truth model def ground_truth(xs): y = (xs ** 2.0) y[xs < 0] = 0 return y

Plot the Data¶

Let's plot the data to see what we are working with.

In [5]:

plt.plot(X, ground_truth(X), 'o')
plt.show()

plt.plot(X, ground_truth(X), 'o') plt.show()

Define and Fit Theorists¶

Next, we initialize each theorist and then train them on the data.

Note that this can take quite some time, especially for the BSR Theorist.

In [6]:

%%capture

#Initiate theorists
darts_theorist = DARTSRegressor()

#Fit theorists
darts_theorist.fit(X,ground_truth(X))

%%capture #Initiate theorists darts_theorist = DARTSRegressor() #Fit theorists darts_theorist.fit(X,ground_truth(X))

Plot Theorists on Data¶

We can then plot each theorist to see how well it recovered the data.

In [7]:

plt.plot(X, ground_truth(X), 'o')
plt.plot(X, darts_theorist.predict(X), alpha = .5, label = 'DARTS Theorist')
plt.legend()
plt.show()

plt.plot(X, ground_truth(X), 'o') plt.plot(X, darts_theorist.predict(X), alpha = .5, label = 'DARTS Theorist') plt.legend() plt.show()

Run and Leverage Samples¶

Now we will get a proposal from the experimentalist as to which datapoints to investigate next. We will retrieve 5 new datapoints in this example.

In [8]:

sampler_proposal = leverage_sample(X, ground_truth(X.ravel()), [darts_theorist], fit = 'both', num_samples = 20, sd=.2)

print('New datapoints:\n' + str(sampler_proposal))

sampler_proposal = leverage_sample(X, ground_truth(X.ravel()), [darts_theorist], fit = 'both', num_samples = 20, sd=.2) print('New datapoints:\n' + str(sampler_proposal))

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

New datapoints:
           0
0   6.464159
1   6.132953
2   4.315494
3   4.850175
4   3.941828
5  -3.477950
6   5.529659
7  -1.218220
8  -0.632884
9  -0.369711
10  1.548444
11  1.985621
12  0.943885
13  1.218785
14  1.440813
15 -2.020381
16  0.510689
17 -2.342074
18 -2.073968
19  2.202574

Plot New Datapoints With Old¶

We can then plot our new datapoints with our previous ones to demonstrate our new dataset of investigation for then next cycle.

In [9]:

plt.plot(X, ground_truth(X), 'o', alpha = .5, label = 'Original Datapoints')
plt.plot(sampler_proposal, ground_truth(sampler_proposal), 'o', alpha = .5, label = 'Proposed Datapoints')
plt.legend()
plt.show()

plt.plot(X, ground_truth(X), 'o', alpha = .5, label = 'Original Datapoints') plt.plot(sampler_proposal, ground_truth(sampler_proposal), 'o', alpha = .5, label = 'Proposed Datapoints') plt.legend() plt.show()

Plot Data with New Models¶

We will now refit our models with the new, extended, dataset to see if it looks any better

In [11]:

#Create new independent variable
original_X = X
X = np.concatenate((X, np.array(sampler_proposal).reshape(-1,1)))
X.reshape(-1).sort(kind='mergesort')

#Refit models
original_darts_theorist = DARTSRegressor()
darts_theorist = DARTSRegressor()
original_darts_theorist.fit(original_X,ground_truth(original_X))
darts_theorist.fit(X,ground_truth(X))

#Plot data and models
plt.plot(X, ground_truth(X), 'o', alpha = .5, label = 'Original Datapoints')
plt.plot(sampler_proposal, ground_truth(sampler_proposal), 'o', alpha = .5, label = 'Proposed Datapoints')
plt.plot(original_X, original_darts_theorist.predict(original_X), alpha = .5, label = 'Original DARTS Theorist')
plt.plot(X, darts_theorist.predict(X), alpha = .5, label = 'New DARTS Theorist')
plt.legend()
plt.show()

#Create new independent variable original_X = X X = np.concatenate((X, np.array(sampler_proposal).reshape(-1,1))) X.reshape(-1).sort(kind='mergesort') #Refit models original_darts_theorist = DARTSRegressor() darts_theorist = DARTSRegressor() original_darts_theorist.fit(original_X,ground_truth(original_X)) darts_theorist.fit(X,ground_truth(X)) #Plot data and models plt.plot(X, ground_truth(X), 'o', alpha = .5, label = 'Original Datapoints') plt.plot(sampler_proposal, ground_truth(sampler_proposal), 'o', alpha = .5, label = 'Proposed Datapoints') plt.plot(original_X, original_darts_theorist.predict(original_X), alpha = .5, label = 'Original DARTS Theorist') plt.plot(X, darts_theorist.predict(X), alpha = .5, label = 'New DARTS Theorist') plt.legend() plt.show()

  0%|          | 0/10 [00:00<?, ?it/s]

  0%|          | 0/10 [00:00<?, ?it/s]

In [ ]: