Source ID and Fragment ID Overhaul

[retrodaredevil]

Source ID and Fragment ID have been around a while. The general idea behind them is this:

• Every SolarThing instance has a unique fragment ID. This helps distinguish a given packet collection from other packet collections that are from different devices
• Most SolarThing instances are configured to use the "default" Source ID. This can be used to separate unrelated devices.

There are a couple of problems with how these are used across SolarThing.

First, Source IDs are not configurable after the fact. This is not good because it means once data is uploaded with a particular source ID, it cannot be changed. Additionally, source IDs introduce unneeded complexity when configuring SolarThing because of how infrequently a value other than "default" is used.

Fragment IDs work pretty well, but the value of the fragment ID actually determines how it is used in SolarThing. This is not well documented, and probably isn't what we want, since it is not changeable after the fact. Its main use is shown here:

solarthing/core/src/main/java/me/retrodaredevil/solarthing/packets/collection/PacketGroups.java

Lines 160 to 190 in cd9f6e2

	public static List<FragmentedPacketGroup> mergePackets(List<? extends InstancePacketGroup> instancePacketGroups, long maxTimeDistance, Long masterIdIgnoreDistance, Comparator<Integer> fragmentIdComparator){
	if (instancePacketGroups.isEmpty()) {
	throw new IllegalArgumentException("instancePacketGroups cannot be empty!");
	}
	Map<Integer, List<InstancePacketGroup>> fragmentMap = new HashMap<>();
	for(InstancePacketGroup packetGroup : instancePacketGroups){ // this for loop initializes fragmentMap
	final int fragmentId = packetGroup.getFragmentId();
	List<InstancePacketGroup> fragmentList = fragmentMap.get(packetGroup.getFragmentId());
	if(fragmentList == null){
	fragmentList = new ArrayList<>();
	fragmentMap.put(fragmentId, fragmentList);
	}
	fragmentList.add(packetGroup);
	}
	final List<Integer> fragmentIds;
	{ // initialize fragmentIds
	SortedSet<Integer> fragmentIdsSet = new TreeSet<>(fragmentIdComparator);
	fragmentIdsSet.addAll(fragmentMap.keySet());
	fragmentIds = new ArrayList<>(fragmentIdsSet); // now this is sorted
	}
	// Create an ordered set so that whenever something is added to it, it gets ordered correctly
	TreeSet<FragmentedPacketGroup> packetGroups = new TreeSet<>(Comparator.comparingLong(PacketGroup::getDateMillis));
	addToPacketGroups(
	maxTimeDistance, masterIdIgnoreDistance,
	Long.MIN_VALUE, Long.MAX_VALUE,
	fragmentIds,
	fragmentMap,
	packetGroups
	);
	return new ArrayList<>(packetGroups);
	}

---

I think the solution should be that Source IDs are removed altogether and replaced with some sort of grouping that can be configured in the metadata of the database. We should find a better solution than mergePackets that doesn't involve selecting the lowest fragment ID to be the master.

Not sure when I'll get around to this, but these are my thoughts on it for now.