beats-like-a-helix
02/25/2022, 6:47 PMwulfcrona
02/28/2022, 12:43 PMlbonini
02/28/2022, 2:19 PMdatajoely
02/28/2022, 2:23 PMafter_pipeline_created
hook that replaces the dataset with a MemoryDataSet dynamically based on the parameter or env variablelbonini
02/28/2022, 2:26 PMbeats-like-a-helix
03/04/2022, 7:17 PMbeats-like-a-helix
03/04/2022, 8:32 PMpython
plots_dict = dict()
for colour in ["blue", "green", "red"]:
plots_dict[f"{colour}.pdf"] = plt.figure()
plt.plot([1, 2, 3], [4, 5, 6], color=colour)
plt.close("all")
dict_plot_writer = MatplotlibWriter(
filepath="matplotlib_dict",
save_args={
# "format": "pdf",
"dpi": 300,
"bbox_inches": "tight",
},
)
dict_plot_writer.save(plots_dict)
pypeaday
03/04/2022, 9:37 PMdesrame
03/04/2022, 10:14 PMdesrame
03/04/2022, 10:15 PMbeats-like-a-helix
03/04/2022, 10:17 PMbeats-like-a-helix
03/04/2022, 10:31 PMyml
power_spectrum_figures:
type: PartitionedDataSet
path: data/07_model_output/figures
dataset:
type: matplotlib.MatplotlibWriter
save_args:
format: pdf
dpi: 300
bbox_inches: tight
filename_suffix: ".pdf"
beats-like-a-helix
03/04/2022, 10:37 PMsh
.
├── README.md
├── conf
├── data
├── docs
├── info.log
├── logs
├── notebooks
├── pyproject.toml
├── setup.cfg
└── src
beats-like-a-helix
03/04/2022, 10:42 PMdesrame
03/04/2022, 10:44 PMdesrame
03/04/2022, 10:45 PMdesrame
03/05/2022, 3:25 AMdesrame
03/05/2022, 3:26 AMdesrame
03/05/2022, 3:28 AM# catalog.yml definition
lstm_base:
type: pandas.SQLTableDataSet
table_name: sometable
credentials: sql1
# credentials.yml in local
sql1:
con: mssql+pyodbc:///?odbc_connect=DRIVER={ODBC+Driver+17+for+SQL+Server};SERVER=someserver;DATABASE=somedatabase;UID=someuser;PWD=somepwd
# .py version working in .py script
conn = \
'DRIVER={ODBC Driver 17 for SQL Server};SERVER=someserver;DATABASE=somedatabase;UID=someuser;PWD=somepassword'
quoted = quote_plus(conn)
new_con = 'mssql+pyodbc:///?odbc_connect={}'.format(quoted)
engine = create_engine(new_con, fast_executemany=True, connect_args={'timeout': 100})
desrame
03/05/2022, 3:28 AMdatajoely
03/05/2022, 11:27 AMdatajoely
03/05/2022, 11:27 AMdesrame
03/05/2022, 6:08 PMdesrame
03/05/2022, 6:10 PMfrom kedro.config import ConfigLoader, MissingConfigException
conf_paths = ["conf/base", "conf/local"]
conf_loader = ConfigLoader(conf_paths)
try:
credentials = conf_loader.get("credentials*", "credentials*/**")
except MissingConfigException:
credentials = {}
credentials
desrame
03/05/2022, 6:11 PMdatajoely
03/05/2022, 6:13 PMdesrame
03/05/2022, 6:13 PMdesrame
03/05/2022, 6:13 PMdatajoely
03/05/2022, 6:13 PMdatajoely
03/05/2022, 6:13 PM