Introduce sample search quality dashboards and supporting data

[epugh]

Description

Introduces new Search Quality dashboards.

This is part one of two. The second part will be some new sample UBI Dashboards.

Issues Resolved

Partly addresses #214

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[epugh]

@gaiksaya when we chatted, I think you pointed me to where I need to add the install script. The original script I used is here, https://github.com/opensearch-project/user-behavior-insights/blob/main/search-quality-evaluation-dashboards/install_dashboards.sh.

Can you remind me how to load the data as part of the nightly?

Once this is in, I'll also open a PR for the UBI dashboards! Under the sample sample-data directory.

[epugh]

not quite sure how to integrate and test the loading script... Something like?

curl -X POST -f 'https://${{needs.validate-and-deploy.outputs.ENDPOINT}}/${{needs.validate-and-deploy.outputs.PLAYGROUND_ID}}/sqe_metrics_sample_data/_bulk?pretty=false&filter_path=-items"' -H 'securitytenant: global' -H 'Content-Type: application/json' -H 'osd-version: ${{inputs.dist_version}}' -H 'osd-xsrf: osd-fetch' -u ${{ secrets.OPENSEARCH_USER }}:${{ secrets.OPENSEARCH_PASSWORD }} --insecure --data-binary @./nightly-playground/sample-data/search-quality-evaluation-dashboards/sample_search_quality_evaluation_data.ndjson

[gaiksaya]

not quite sure how to integrate and test the loading script... Something like?

curl -X POST -f 'https://${{needs.validate-and-deploy.outputs.ENDPOINT}}/${{needs.validate-and-deploy.outputs.PLAYGROUND_ID}}/sqe_metrics_sample_data/_bulk?pretty=false&filter_path=-items"' -H 'securitytenant: global' -H 'Content-Type: application/json' -H 'osd-version: ${{inputs.dist_version}}' -H 'osd-xsrf: osd-fetch' -u ${{ secrets.OPENSEARCH_USER }}:${{ secrets.OPENSEARCH_PASSWORD }} --insecure --data-binary @./nightly-playground/sample-data/search-quality-evaluation-dashboards/sample_search_quality_evaluation_data.ndjson

Right. We are doing something similar here: https://github.com/opensearch-project/opensearch-devops/blob/main/.github/workflows/nightly-playground-deploy.yml#L140-L144
We can add this to same block.

[epugh]

@gaiksaya I just added the two curl commands similar to the ones you linked. I don't have any way to test this (should I be able to run it locally???). I'd love to get a pairing session with you today or tomorrow if possible. We have a demo on Thursday and I'd love to be able to show this off!

[gaiksaya]

@gaiksaya I just added the two curl commands similar to the ones you linked. I don't have any way to test this (should I be able to run it locally???). I'd love to get a pairing session with you today or tomorrow if possible. We have a demo on Thursday and I'd love to be able to show this off!

Hi @epugh
Sorry was out of office for a week. I tried running command and received 404. I believe we need to create the index sqe_metrics_sample_data first right?

[epugh]

@gaiksaya I just added the two curl commands similar to the ones you linked. I don't have any way to test this (should I be able to run it locally???). I'd love to get a pairing session with you today or tomorrow if possible. We have a demo on Thursday and I'd love to be able to show this off!

Hi @epugh Sorry was out of office for a week. I tried running command and received 404. I believe we need to create the index sqe_metrics_sample_data first right?

You are correct. For some reason I thought we created it "on the fly"... Let me tweak the PR and have something for you tomorrow with that!

[epugh]

FYI, we ended up redoing a lot of this work in the past few months, taking lessons learned, so this went on hiatus... I may close it in the next few weeks...