hive_metastore_client package

Subpackages

Submodules

Hive Metastore Client main class.

class hive_metastore_client.hive_metastore_client.HiveMetastoreClient(host: str, port: int = 9083)

Bases: Client

User main interface with the metastore server methods.

COL_TYPE_INCOMPATIBILITY_DISALLOW_CONFIG = 'hive.metastore.disallow.incompatible.col.type.changes'
add_columns_to_table(db_name: str, table_name: str, columns: List[FieldSchema]) → None

Adds columns to a table.

Parameters

  • db_name – database name of the table

  • table_name – table name

  • columns – columns to be added to the table

add_partitions_if_not_exists(db_name: str, table_name: str, partition_list: List[Partition]) → None

Add partitions to a table if it does not exist.

If a partition is added twice, the method handles the

AlreadyExistsException, not raising the exception.

Parameters

  • db_name – database name where the table is at

  • table_name – table name which the partitions belong to

  • partition_list – list of partitions to be added to the table

add_partitions_to_table(db_name: str, table_name: str, partition_list: List[Partition]) → None

Add partitions to a table.

If any partition of partition_list already exists, an

AlreadyExistsException, will be thrown and no partition will be added.

Parameters

  • db_name – database name where the table is at

  • table_name – table name which the partitions belong to

  • partition_list – list of partitions to be added to the table

bulk_drop_partitions(db_name: str, table_name: str, partition_list: List[List[str]], delete_data: bool = False) → None

Drops the partitions values from the partition list.

This methods simulates a bulk drop for the user, since the server only

supports an unitary drop.

If some partition cannot be dropped an exception will be thrown in the

end of execution.

Parameters

  • db_name – database name of the table

  • table_name – table name

  • partition_list – the partitions to be dropped

  • delete_data – indicates whether the data respective to the partition should be dropped in the source.

Raises

NoSuchObjectException

close() → None

Closes the connection with the Thrift server.

create_database_if_not_exists(database: Database) → None

Creates the table in Hive Metastore if it does not exist.

Since hive metastore server and thrift mapping do not have the option

of checking if the database does not exist, this method simulates this this behavior.

Parameters

database – the database object

create_external_table(table: Table) → None

Creates an external table in Hive Metastore.

When a table is created with tableType default (None) or equal to

EXTERNAL_TABLE there is an error in the server that creates the table as a MANAGED_TABLE.

This method enforces the parameter EXTERNAL=TRUE so the table is

created correctly.

Parameters

table – the table object

drop_columns_from_table(db_name: str, table_name: str, columns: List[str]) → None

Drops columns from a table.

It encapsulates the logic of calling alter table with removed columns from the list of columns, since hive does not have a drop command.

Parameters

  • db_name – database name of the table

  • table_name – table name

  • columns – names of the columns to be dropped from the table

get_partition_keys(db_name: str, table_name: str) → List[Tuple[str, str]]

Gets the partition keys from a table as a tuple: (name, type).

An empty list will be returned when no table is found or when the table has no partitions.

Parameters

  • db_name – database name where the table is at

  • table_name – table name which the partition keys belong to

get_partition_keys_names(db_name: str, table_name: str) → List[str]

Gets the partition keys names from a table.

An empty list will be returned when no table is found or when the table has no partitions

Parameters

  • db_name – database name where the table is at

  • table_name – table name which the partition keys belong to

get_partition_keys_objects(db_name: str, table_name: str) → List[FieldSchema]

Gets the partition keys objects, containing the metadata, from a table.

An empty list will be returned when no table is found or when the table has no partitions

Parameters

  • db_name – database name where the table is at

  • table_name – table name which the partition keys belong to

get_partition_values_from_table(db_name: str, table_name: str) → List[List[str]]

Gets the partition names from a table.

It automatically fetches the table’s partition keys.

An empty list will be returned when no table is found or when the table has no partitions.

Parameters

  • db_name – database name where the table is at

  • table_name – table name which the partitions belong to

open() → HiveMetastoreClient

Opens the connection with the Thrift server.

Returns

HiveMetastoreClientConnector instance

Module contents

Hive Metastore Client.