hive_metastore_client package¶
Submodules¶
Hive Metastore Client main class.
-
class
hive_metastore_client.hive_metastore_client.
HiveMetastoreClient
(host: str, port: int = 9083)¶ Bases:
thrift_files.libraries.thrift_hive_metastore_client.ThriftHiveMetastore.Client
User main interface with the metastore server methods.
-
COL_TYPE_INCOMPATIBILITY_DISALLOW_CONFIG
= 'hive.metastore.disallow.incompatible.col.type.changes'¶
-
add_columns_to_table
(db_name: str, table_name: str, columns: List[thrift_files.libraries.thrift_hive_metastore_client.ttypes.FieldSchema]) → None¶ Adds columns to a table.
- Parameters
db_name – database name of the table
table_name – table name
columns – columns to be added to the table
-
add_partitions_if_not_exists
(db_name: str, table_name: str, partition_list: List[thrift_files.libraries.thrift_hive_metastore_client.ttypes.Partition]) → None¶ Add partitions to a table if it does not exist.
- If a partition is added twice, the method handles the
AlreadyExistsException, not raising the exception.
- Parameters
db_name – database name where the table is at
table_name – table name which the partitions belong to
partition_list – list of partitions to be added to the table
-
add_partitions_to_table
(db_name: str, table_name: str, partition_list: List[thrift_files.libraries.thrift_hive_metastore_client.ttypes.Partition]) → None¶ Add partitions to a table.
- If any partition of partition_list already exists, an
AlreadyExistsException, will be thrown and no partition will be added.
- Parameters
db_name – database name where the table is at
table_name – table name which the partitions belong to
partition_list – list of partitions to be added to the table
-
bulk_drop_partitions
(db_name: str, table_name: str, partition_list: List[List[str]], delete_data: bool = False) → None¶ Drops the partitions values from the partition list.
- This methods simulates a bulk drop for the user, since the server only
supports an unitary drop.
- If some partition cannot be dropped an exception will be thrown in the
end of execution.
- Parameters
db_name – database name of the table
table_name – table name
partition_list – the partitions to be dropped
delete_data – indicates whether the data respective to the partition should be dropped in the source.
- Raises
NoSuchObjectException
-
close
() → None¶ Closes the connection with the Thrift server.
-
create_database_if_not_exists
(database: thrift_files.libraries.thrift_hive_metastore_client.ttypes.Database) → None¶ Creates the table in Hive Metastore if it does not exist.
- Since hive metastore server and thrift mapping do not have the option
of checking if the database does not exist, this method simulates this this behavior.
- Parameters
database – the database object
-
create_external_table
(table: thrift_files.libraries.thrift_hive_metastore_client.ttypes.Table) → None¶ Creates an external table in Hive Metastore.
- When a table is created with tableType default (None) or equal to
EXTERNAL_TABLE there is an error in the server that creates the table as a MANAGED_TABLE.
- This method enforces the parameter EXTERNAL=TRUE so the table is
created correctly.
- Parameters
table – the table object
-
drop_columns_from_table
(db_name: str, table_name: str, columns: List[str]) → None¶ Drops columns from a table.
It encapsulates the logic of calling alter table with removed columns from the list of columns, since hive does not have a drop command.
- Parameters
db_name – database name of the table
table_name – table name
columns – names of the columns to be dropped from the table
-
get_partition_keys
(db_name: str, table_name: str) → List[Tuple[str, str]]¶ Gets the partition keys from a table as a tuple: (name, type).
An empty list will be returned when no table is found or when the table has no partitions.
- Parameters
db_name – database name where the table is at
table_name – table name which the partition keys belong to
-
get_partition_keys_names
(db_name: str, table_name: str) → List[str]¶ Gets the partition keys names from a table.
An empty list will be returned when no table is found or when the table has no partitions
- Parameters
db_name – database name where the table is at
table_name – table name which the partition keys belong to
-
get_partition_keys_objects
(db_name: str, table_name: str) → List[thrift_files.libraries.thrift_hive_metastore_client.ttypes.FieldSchema]¶ Gets the partition keys objects, containing the metadata, from a table.
An empty list will be returned when no table is found or when the table has no partitions
- Parameters
db_name – database name where the table is at
table_name – table name which the partition keys belong to
-
get_partition_values_from_table
(db_name: str, table_name: str) → List[List[str]]¶ Gets the partition names from a table.
It automatically fetches the table’s partition keys.
An empty list will be returned when no table is found or when the table has no partitions.
- Parameters
db_name – database name where the table is at
table_name – table name which the partitions belong to
-
open
() → hive_metastore_client.hive_metastore_client.HiveMetastoreClient¶ Opens the connection with the Thrift server.
- Returns
HiveMetastoreClientConnector instance
-
Module contents¶
Hive Metastore Client.