[pull] memtable_as_log_index from topling:memtable_as_log_index#4
Open
pull[bot] wants to merge 94 commits into
Open
[pull] memtable_as_log_index from topling:memtable_as_log_index#4pull[bot] wants to merge 94 commits into
pull[bot] wants to merge 94 commits into
Conversation
This makes ToplingDB Web auto refresh pretty
rockeet
force-pushed
the
memtable_as_log_index
branch
2 times, most recently
from
9b42dd6 to
5c00a65
Compare
rockeet
force-pushed
the
memtable_as_log_index
branch
from
5c00a65 to
68fbc5f
Compare
And strip db_bench in build-trial.sh
rockeet
force-pushed
the
memtable_as_log_index
branch
from
538e4a3 to
b0622bb
Compare
rockeet
force-pushed
the
memtable_as_log_index
branch
from
b0622bb to
1aa07fa
Compare
rockeet
force-pushed
the
memtable_as_log_index
branch
from
70b0c77 to
8a40448
Compare
rockeet
force-pushed
the
memtable_as_log_index
branch
from
8a40448 to
355304b
Compare
rockeet
force-pushed
the
memtable_as_log_index
branch
from
e4ecd25 to
0892907
Compare
rockeet
force-pushed
the
memtable_as_log_index
branch
from
da95133 to
c78a892
Compare
…, slice_transform, and filter_policy
Provide extern "C" registration functions enabling C-language plugin authors to
register their custom plugins with the side plugin framework. Five plugin types
are covered:
- comparator: bare const pointer (PluginFactory<const Comparator*>)
- merge_operator: shared_ptr (PluginFactory<std::shared_ptr<MergeOperator>>)
- compaction_filter_factory: shared_ptr (ditto for CompactionFilterFactory)
- slicetransform: shared_ptr<const SliceTransform>
- filterpolicy: shared_ptr<const FilterPolicy>
Each has a register/unregister pair.
Two internal template helpers bridge the C/C++ boundary:
- side_plugin_register_raw_ptr_plugin<Object, FFI_Object>
wraps a C creator (returning FFI_Object*) into a PluginFactory AcqFunc that
returns Object*. Used for comparator, which the framework stores as bare
const pointer.
- side_plugin_register_shared_ptr_plugin<Object, FFI_Object>
likewise wraps a C creator, then wraps the returned pointer in a
shared_ptr<Object>. The C bridge objects (rocksdb_mergeoperator_t, etc.)
have proper destructors that invoke the user-provided cleanup callback,
so normal shared_ptr deletion is safe — no intentional leak needed.
Each wrapper lambda converts the C++ SidePluginRepo reference back to a C
side_plugin_repo_t* (validated by static_assert that the C struct has the C++
object as its first member via offsetof == 0), and serializes the C++ json to a
C string at the FFI boundary.
rockeet
force-pushed
the
memtable_as_log_index
branch
from
4d817eb to
9f0964f
Compare
Include side_plugin_tpl_inst.cc: add explicit_instantiate_serde(FilterPolicy)
Add FFI_SerDe/FFI_WebManip for distributed compaction serde and webview bridge. Introduce side_plugin_ex_vtab_t for C-side vtable. Update all register/unregister functions to support extended vtab (serde + web).
Allow closing a single DB instance managed by the repo.
rockeet
force-pushed
the
memtable_as_log_index
branch
from
f07d1df to
207a070
Compare
rockeet
force-pushed
the
memtable_as_log_index
branch
2 times, most recently
from
720d183 to
80f7c95
Compare
These two methods let users keep their existing DBOptions and CFOptions construction code while selectively applying options from a SidePluginRepo YAML/JSON config. Users load a config file into a SidePluginRepo, then call side_plugin_db_options_update_from(opt, repo, "name"); side_plugin_cf_options_update_from(opt, repo, "name"); to overlay the repo's named options onto hand-crafted Options objects. This migration path preserves all ToplingDB engine optimizations (MemTable, SST formats, etc.) but foregoes SidePlugin features that depend on full SidePluginRepo-managed lifecycle: Web UI observability, Prometheus integration, online config hot-update, and distributed compaction.
rockeet
force-pushed
the
memtable_as_log_index
branch
from
80f7c95 to
bf54b9c
Compare
Also override CFOptions in CreateColumnFamily, override options through MaybeOptionsUpdateFrom() If import the conf file failed, the process dies immediately
Under TOPLINGDB_EASY_MIGRATE_CONF, db-path components are treated as
a namespace hierarchy for config lookup, from most specific (full path)
to most general ("default"). CFOptions now follow the same path-prefix
walk with bare-name and "default" fallbacks that DBOptions always had.
Design rationale: https://github.com/topling/rockside/wiki/Easy-Migrate-Without-Code-Change
rockeet
force-pushed
the
memtable_as_log_index
branch
from
b1128dc to
18084ab
Compare
- Call MaybeRetainDB via ROCKSDB_SCOPE_EXIT after successful DBImpl::Open, ReadOnly, and Secondary opens. - Call MaybeForgetDB at the start of ~DBImpl. - Clear *dbptr before attaching scope exit to avoid retaining on early validation failure; setting *dbptr = nullptr here also fixes an obscure upstream bug where the output pointer could be left uncleared when validation fails before the later assignment. - Bump rockside for EasyMigrate HTTP gating and retain/forget implementation.
The MaybeRetainDB fix in the rockside submodule needs to read persist_stats_cf_handle_ from a DB* pointer, but it is a private member with no accessor. Add a public accessor and a wrapper function, and update the submodule to use them. The submodule fix replaces dangling default_cfh and stat_cfh handles with live pointers before storing them in the sideplugin repo, since RocksDB deletes the original handles on simple Open.
Track column family lifecycle in the sideplugin repo during easy migrate: register CF in CreateColumnFamily via scope-exit guard, and unregister in DropColumnFamily before the handle becomes invalid.
Pulls in the yaml config change and the MaybeRetainCF/MaybeForgetCF implementation in the submodule.
Expose ReadOptions::internal_is_in_pinning_section through the C API, needed by Rust bindings for ReadOptionsScopePinIfNotPinned guard.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )