datajoely
01/13/2022, 10:17 AMdatajoely
01/13/2022, 10:18 AMcatalog.add()
ChainYo
01/13/2022, 10:20 AMChainYo
01/13/2022, 10:20 AMChainYo
01/13/2022, 10:21 AMdatajoely
01/13/2022, 10:25 AMdatajoely
01/13/2022, 10:26 AMdatajoely
01/13/2022, 10:26 AMdatajoely
01/13/2022, 10:49 AMggerog
01/13/2022, 3:42 PMdatajoely
01/13/2022, 4:08 PMlogging.yml
to configure what the various loggers do.
2) In python world you can grab a logger and start using it via logging.getLogger(name)
datajoely
01/13/2022, 4:08 PMname
of the the journal logger is, maybe kedro.journal
but not 100% sureggerog
01/13/2022, 4:13 PMhandlers:
console:
class: logging.StreamHandler
level: INFO
formatter: simple
stream: ext://sys.stdout
info_file_handler:
class: logging.handlers.RotatingFileHandler
level: INFO
formatter: simple
filename: logs/info.log
maxBytes: 10485760 # 10MB
backupCount: 20
encoding: utf8
delay: True
error_file_handler:
class: logging.handlers.RotatingFileHandler
level: ERROR
formatter: simple
filename: logs/errors.log
maxBytes: 10485760 # 10MB
backupCount: 20
encoding: utf8
delay: True
journal_file_handler:
class: kedro.versioning.journal.JournalFileHandler
level: INFO
base_dir: logs/journals
formatter: json_formatter
datajoely
01/13/2022, 4:14 PMdatajoely
01/13/2022, 4:15 PMggerog
01/13/2022, 4:17 PMdatajoely
01/13/2022, 4:17 PMdatajoely
01/13/2022, 4:18 PMggerog
01/13/2022, 4:40 PMformatters:
simple:
format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
datajoely
01/13/2022, 5:03 PMdatajoely
01/13/2022, 5:03 PMdatajoely
01/13/2022, 5:04 PMlogging.config.dictConfig
ggerog
01/13/2022, 5:24 PMggerog
01/14/2022, 12:54 PMkedro run --to-nodes=<node name>
, and go step by step.ggerog
01/14/2022, 12:55 PMdatajoely
01/14/2022, 12:56 PMkedro run --node
etc.
2. Using a proper debugger in VS Code or PyCharmdatajoely
01/14/2022, 12:56 PMdatajoely
01/14/2022, 12:57 PMggerog
01/14/2022, 1:02 PMwith KedroSession.create("library") as session:
context = session.load_context()
print(context.params)
session.run(<node1 options>)
run.node1_tests()
session.run(<node2 options>)
... so on
Does that seem reasonable?datajoely
01/14/2022, 1:03 PM