Data reliability vs. offline capability

[fschein]

Hi brothers,

I wanted to open a discussion about the current data synchronization strategy used in the Organized app. As we move towards a more polished and stable release, I believe this is a critical area impacting adoption and user trust.

1. Current Approach and Observed Issues
   The current system seems to be based on a sync-on-demand model, likely chosen to support offline usability, which is a huge benefit for our users.

However, we are seeing significant friction and issues:

• Frequent Sync Errors: Users are constantly encountering sync conflicts or failure messages, which detracts from a seamless experience.
• Data Integrity Concerns: If a user makes a change offline, another user might not receive that update until much later (or even experience a sync conflict), leading to potential confusion and data inconsistency across devices. This creates a low sense of data security and reliability.
• Slow Adoption: These reliability issues (sync errors, lack of instant feedback) are, in my view, contributing to slower user adoption and perceived "lack of polish," even if the feature set is excellent.

2. A Call for Reassessment (Considering Local Context)
   While offline capability is essential, I think we need to revisit the cost-benefit analysis of the current strategy, especially when considering the majority of our user base.

In Brazil, mobile internet access is widespread and reliable for most users. It's highly probable that the vast majority of our users are online most of the time via 3G/4G/5G connections. Therefore, prioritizing the current sync model solely to benefit the minority of truly offline users might be negatively impacting the majority who could benefit from a more stable, real-time sync experience.

A data synchronization model that frequently fails or is confusing to the user is worse than one that is always up-to-date and reliable (even if it requires a temporary "read-only" state when truly offline).

I propose we discuss a move towards a more robust, possibly cloud-centric, or real-time synchronization model.

Possible alternatives to explore:

• Real-time/Near Real-time Synchronization (e.g., using a managed service): When connected, changes are pushed immediately.
• Conflict Resolution: Implementing clearer and more intelligent logic for resolving conflicts when they do occur.
• "Eventually Consistent" with Clear UX: If we keep the current model, the UX must be absolutely clear about which changes are local vs. synced, and provide a guaranteed mechanism for conflict resolution that the user can trust.

I'm eager to hear the reasoning behind the original choice and the challenges in implementing a more reliable system. What are your thoughts on shifting the priority from maximum offline-use to maximum data integrity and reliability, given our users' widespread internet access?

Thank you for considering this important architectural point.

[rhahao]

Hi @fschein, thanks for this feedback. We will take a look at this, and get back to you.

[nobodyzero1]

Hi @rhahao and @fschein !
I have also given some thought to the synchronization aspect.

I cannot comment on the technical difficulties or implementation details regarding synchronization due to a lack of experience. My focus is more on how to best implement the organization's instructions regarding data privacy while simultaneously allowing for the most convenient use of the app possible. My thoughts are based on the rules known to me in Germany.

Known Guidelines (German Context)

• Confidential data must not be stored on cloud storage systems. This specifically includes Field Service Reports.
• Only data that is also displayed on the notice board may be stored on cloud storage.
• Publisher Record Cards should be requested from the Secretary when needed and deleted after review.
• Contact details should always be kept up-to-date and sent to all elders.

Conclusions for the App Logic

Based on these principles, I would suggest the following access/sync structure:

1. Schedules: Meeting schedules and group overviews can be viewed by all app users in a congregation. Synchronization here does not need to meet special privacy requirements.
2. Contact Data: Should be available to all elders. All other publishers should only be able to see and edit their own contact data.
3. Field Service Reports: Since reports are collected by Group Overseers (or their assistants), they should be able to see the reports or the status of the publishers in their group for the current/last month.
   • Other elders should not see this.
   • Group Overseers should not see the historical history of the reports (based on the rule that cards are requested on demand and then deleted).
4. Secretary: Only the Secretary should see all data. Based on the instructions, it seems intended that the majority of the historical data remains solely with them.

Cloud Storage & Encryption

Since the data is End-to-End Encrypted (E2EE), I view the requirement "not to store data in the cloud" as being met, despite the synchronization.

However, I think there should be granular controls in the settings regarding which data is synced at all and with whom. A body of elders might decide that the data held by the Secretary should remain stored strictly locally on their device, completely without synchronization.

Transparency / Feature Idea

To give users greater confidence regarding data privacy, it would be great (perhaps in a separate menu item) to have the ability to inspect the raw text/payload being sent during synchronization.

• This would make it visible to the user that the data contains no clear text.
• Perhaps additional information could be listed, such as "Who holds the keys to decrypt this specific data packet?".

Conclusion

Based on these requirements, I (as a layperson) would say there shouldn't be major issues with the current synchronization approach regarding conflicts, as most data types are usually edited by only one specific person at a time (e.g., the Group Overseer edits their group's report, the scheduler edits the schedule).

[rhahao]

Hello @nobodyzero1, thank you for taking the time to share your thoughts and for explaining your perspective so clearly, it’s really appreciated.

I agree that the idea of having more granularity over what gets synced (and how) is a good approach. It’s definitely an interesting direction to think about when balancing offline capability with data reliability.

At the moment, though, I can’t promise that something like this will be added anytime soon. Still, feedback like yours is valuable and helps shape how we think about the app’s evolution.

Thanks.