Description
Expose notebook and application id as variables that can be used in logging
Use case or motivation behind the feature request
Having logs like
INFO [2024-09-16 12:05:50,398] ({spark-listener-group-streams} Logging.scala[logInfo]:60) - Asked to cancel job group 9dfa0397-114e-494b-a43e-1563a5a4e9c2
INFO [2024-09-16 12:05:50,398] ({spark-listener-group-streams} Logging.scala[logInfo]:60) - Query `application_1726477412269_0001-0` [id = 737e5440-ae1f-4b4b-9a18-4a30d628b1ea, runId = 9dfa0397-114e-494b-a43e-1563a5a4e9c2] was stopped
INFO [2024-09-16 12:05:50,399] ({spark-listener-group-streams} DPLStreamingQueryListener.java[onQueryTerminated]:82) - Query terminated: 737e5440-ae1f-4b4b-9a18-4a30d628b1eaisn't very useful if I don't know the original notebook id to trace it back to.
Make it so that logging has a variable available so we could have logs like
INFO [2024-09-16 12:05:50,398] *[application_1726477412269_0001] [2JSMD5SAE]* ({spark-listener-group-streams} Logging.scala[logInfo]:60) - Asked to cancel job group 9dfa0397-114e-494b-a43e-1563a5a4e9c2
INFO [2024-09-16 12:05:50,398] *[application_1726477412269_0001] [2JSMD5SAE]* ({spark-listener-group-streams} Logging.scala[logInfo]:60) - Query `application_1726477412269_0001-0` [id = 737e5440-ae1f-4b4b-9a18-4a30d628b1ea, runId = 9dfa0397-114e-494b-a43e-1563a5a4e9c2] was stopped
INFO [2024-09-16 12:05:50,399] *[application_1726477412269_0001] [2JSMD5SAE]* ({spark-listener-group-streams} DPLStreamingQueryListener.java[onQueryTerminated]:82) - Query terminated: 737e5440-ae1f-4b4b-9a18-4a30d628b1eaRelated issues
Additional context
Description
Expose notebook and application id as variables that can be used in logging
Use case or motivation behind the feature request
Having logs like
INFO [2024-09-16 12:05:50,398] ({spark-listener-group-streams} Logging.scala[logInfo]:60) - Asked to cancel job group 9dfa0397-114e-494b-a43e-1563a5a4e9c2 INFO [2024-09-16 12:05:50,398] ({spark-listener-group-streams} Logging.scala[logInfo]:60) - Query `application_1726477412269_0001-0` [id = 737e5440-ae1f-4b4b-9a18-4a30d628b1ea, runId = 9dfa0397-114e-494b-a43e-1563a5a4e9c2] was stopped INFO [2024-09-16 12:05:50,399] ({spark-listener-group-streams} DPLStreamingQueryListener.java[onQueryTerminated]:82) - Query terminated: 737e5440-ae1f-4b4b-9a18-4a30d628b1eaisn't very useful if I don't know the original notebook id to trace it back to.
Make it so that logging has a variable available so we could have logs like
Related issues
Additional context