Different labels needed for second line graph, chart js

I have two data sets of values and labels. values1 = [123.45,23423.2,.3423,…] for about 50,000 values (coming programmatically)

labels1 = [1,2,3,4,…,length of values1]

and then I have values2 = [75.633,124.34,76.45,89.00] which is the peak values for the above portions of values1

labels2 = the position where the peak was found. If the 7th item is the peak, then labels2[0] = 7, etc. etc. for the 4 peak values I have.

My JS/HTML looks like this

    <script>

var line_chart = {
  type: 'line',
  data: {
    labels: {{ labels | safe }},
    datasets: [{
        label: 'Sweep data',
        data: {{ values | safe }},
        backgroundColor: 'red',
        borderWidth: 1
      },
      {
        label: 'Peak data',
        data: [{
          x: {{ values2 | safe }},
          y: {{ labels2 | safe }}
        }],
        backgroundColor: 'black',
        borderWidth: 1,
        showLine: true
      }
    ]
  },
  options: {
  }
}

var context = document.getElementById('myChart').getContext('2d');
new Chart(context, line_chart);

I was told that in order to have two different datasets that are graphed on the same chart, I would have to specify which x and y correlate to each value. I’m not sure how to do this since the second list of values and labels is coming programmatically from my main.py.

What I’m trying to do is have the 50,000 values in values1 be graphed in red, and then the 4 peak values be graphed in black on top of those.