Freeside - User contributions [en]

Freeside:4:Documentation

2017-02-08T05:43:24Z

Mark: add link to billing internals

= Note =

Some documentation links point to earlier versions until things are fully moved over. The information should still be applicable.

= Installation and upgrades =

== Backend Installation ==

=== Packages ===

* [[Freeside:4:Documentation:InstallingOnDebian8|Debian 8 "jessie"]]

=== Source ===

* [[Freeside:4:Documentation:Installation|New Installation]]
* [[Freeside:3:Documentation:RT Installation|Integrated RT Installation]]
* [[Freeside:3:Documentation:Torrus Installation|Integrated Torrus Installation]]

== Signup and Self-service installation ==

* [[Freeside:3:Documentation:Self-Service Installation|Signup/Self-service Installation]]
* [[Freeside:3:Documentation:Self-Service access without service|Self-Service access without package/service]]
* [[Freeside:3:Documentation:NG_Self-Service| Next generation self-service portal]] (Work in progress)

== Upgrading ==

* [[Freeside:4:Documentation:Upgrading|Upgrading from 3.3 or later to 4.x]]
* [[Freeside:3:Documentation:Upgrading|Upgrading between 4.x versions]]

= Users =

* [[Freeside:3:Documentation:User|User's Guide]] (help wanted)
* [http://www.sisd.com/~ivan/freeside-slides Training presentation slides]

== Features in v3 and v4 that somehow got documented here ==

* [[Freeside:4:Documentation:Appointments]]

= Administrator =

== Reference ==

* [[Freeside:3:Documentation:Administration|Administrator's Guide]]
* [[:Category:Freeside:1.9:Documentation:Template|Templates]] used and their fill in variables.

== New features in v4 ==

* [[Freeside:4:Documentation:Administrator:Multi-currency|Multi-currency]]
* Emails triggered by system log events can be set up at Configuration -> Miscellaneous -> System log emails

== Features in v3 and v4 that somehow got documented here ==

* [[Freeside:4:Documentation:Administrator:Fees|Automated fees]]
* [[Freeside:4:Documentation:Cacti|Cacti Integration]]
* [[Freeside:4:Documentation:MagicMail|MagicMail Integration]]

== Miscellaneous ==

* Some old contributed notes on [[Freeside:Documentation:DisasterRecovery|Disaster Recovery]]

= Developer =

== Reference ==

* [[Freeside:3:Documentation:Developer|Developer's Guide]]
* [[Freeside:3:Documentation:Billing_Internals|Billing Internals]] (walkthrough of the core invoice generation code)

== New features in v4 ==

* [[Freeside:4:Documentation:Developer:Authentication_Plugins|Authentication Plugins]]
* [[Freeside:4:Documentation:TaxEngine|Tax Engines]]

= Accounting Guide =

*[[Freeside:Documentation:Accounting|Accounting Guide]]

= Misc old stuff =

* [[Freeside:Documentation:FAQ|Frequently Asked Questions]]
* [[3rd_party_software|Third-party software]]
* [[Freeside:2.1:Documentation:Administration:VoIP:Timed_Rates|Feature: Timed Rates]]
* [[Freeside:2.1:Documentation:Administration:RT_Workflow|RT workflow features]]
* [[Freeside:2.1:Documentation:Administration:Tips_and_Tricks|Tips and Tricks]]

= Changelog =

* [[Freeside:4.0:Changelog|4.0 Changelog]]
* [[Freeside:4.1:Changelog|4.1 Changelog]]
* [[Freeside:4.2:Changelog|4.2 Changelog]]

Freeside:3:Documentation:Developer:Billing Internals

2017-02-08T05:41:47Z

Mark:

= FS::cust_main::Billing::bill =
This is the main method for generating invoices. It's called on a single FS::cust_main, finds any packages that are due to be billed, and creates one or more invoices with the package charges.

It does not handle sending the invoice to the customer, charging their credit card, suspending packages if they're overdue, or anything else of that kind. Those are events run from collect().

== Initial setup ==
Grab the customer and options. Immediately exit if the customer is “complimentary”. Set the debug level, create a log context, and write a log entry.

my( $self, %options ) = @_;

return '' if $self->complimentary eq 'Y';

local($DEBUG) = $FS::cust_main::DEBUG if $FS::cust_main::DEBUG > $DEBUG;
my $log = FS::Log->new('FS::cust_main::Billing::bill');
my %logopt = (object => $self);

$log->debug('start', %logopt);
warn "$me bill customer ". $self->custnum. "\n"
if $DEBUG;

Set the billing clock. $time is the date we are “billing on”. For example, freeside-daily -y (bill packages a few days in the future so that invoices can be mailed out early) sets $time. This is almost, but not exactly, the same as changing the system clock.

$invoice_time is the date that will appear on invoices. Normally it's the same as $time; the -n option overrides that and forces it to be the real system time.

$cmp_time is the “comparison time”: a package gets billed if its next-bill date is <= $cmp_time. This is the same as $time unless '''next-bill-ignore-time''' is set, in which case we use the end of the day. Note that this does ''not'' affect certain other important time values such as prorate cutoffs.

Also parse the 'not_pkgpart' option to exclude specific packages from billing.

my $time = $options{'time'} || time;
my $invoice_time = $options{'invoice_time'} || $time;

my $cmp_time = ( $conf->exists('next-bill-ignore-time')
? day_end( $time )
: $time
);

$options{'not_pkgpart'} ||= {};
$options{'not_pkgpart'} = { map { $_ => 1 }
split(/\s*,\s*/, $options{'not_pkgpart'})
}
unless ref($options{'not_pkgpart'});

Block out interruptions, start a transaction, and lock the customer record (so that packages aren't ordered or status-changed, payments aren't recorded, etc. while doing this). If anything goes wrong during billing, we will rollback and return an error.

local $SIG{HUP} = 'IGNORE';
local $SIG{INT} = 'IGNORE';
local $SIG{QUIT} = 'IGNORE';
local $SIG{TERM} = 'IGNORE';
local $SIG{TSTP} = 'IGNORE';
local $SIG{PIPE} = 'IGNORE';

my $oldAutoCommit = $FS::UID::AutoCommit;
local $FS::UID::AutoCommit = 0;
my $dbh = dbh;

$log->debug('acquiring lock', %logopt);
warn "$me acquiring lock on customer ". $self->custnum. "\n"
if $DEBUG;

$self->select_for_update; #mutex

== Run pre-billing events. ==
Certain billing event actions are always run just before billing, because they affect the charges that will appear on the bill. These are:

* pkg_discount
* cust_pkg_fee, cust_fee, and pkg_fee
* The old “fee” event that created fees as one-time charges.

In this section we call do_cust_event() with a 'stage' parameter to specify only the pre-billing events.

$log->debug('running pre-bill events', %logopt);
warn "$me running pre-bill events for customer ". $self->custnum. "\n"
if $DEBUG;

my $error = $self->do_cust_event(
'debug' <nowiki>=> ( $options{'debug'} || 0 ),</nowiki>
'time' <nowiki>=> $invoice_time,</nowiki>
'check_freq' => $options{'check_freq'},
'stage' <nowiki>=> 'pre-bill',</nowiki>
)
unless $options{no_commit};
if ( $error ) {
$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return $error;
}

$log->debug('done running pre-bill events', %logopt);
warn "$me done running pre-bill events for customer ". $self->custnum. "\n"
if $DEBUG;

== Identify billable packages and add-on packages. ==
Set up space for the line items (%cust_bill_pkg), setup and recur totals, and a tax engine. Every time we add a line item to the invoice, we must push it to the line item array, add its setup and recur charges to the totals, and send it to the tax engine.

A single call to bill() can generate multiple invoices. Packages can be eligible or ineligible for automatic payment; that's implemented by splitting non-auto-payment packages off onto a separate invoice. So we'll (potentially) make two invoices here, and need a line items array, subtotals, etc. for each of them. If any package has the “separate_bill” flag, we'll make an additional invoice just for that package.

Also, if the set of packages to bill wasn't specified ('pkg_list'; it's usually not), then use all of the customer's non-canceled packages. We will check later for whether they're due for billing; that's complicated.

<nowiki>#keep auto-charge and non-auto-charge line items separate</nowiki>
my @passes = ( '', 'no_auto' );

my %cust_bill_pkg = map { $_ => [] } @passes;

<nowiki>###</nowiki>
<nowiki># find the packages which are due for billing, find out how much they are</nowiki>
<nowiki># & generate invoice database.</nowiki>
<nowiki>###</nowiki>

my %total_setup <nowiki>= map { my $z = 0; $_ => \$z; } @passes;</nowiki>
my %total_recur <nowiki>= map { my $z = 0; $_ => \$z; } @passes;</nowiki>

my @precommit_hooks = ();

$options{'pkg_list'} ||= [ $self->ncancelled_pkgs ]; <nowiki>#param checks?</nowiki>

my %tax_engines;
my $tax_is_batch = '';
foreach (@passes) {
$tax_engines{$_} = FS::TaxEngine->new(cust_main <nowiki>=> $self,</nowiki>
invoice_time => $invoice_time,
cancel <nowiki>=> $options{cancel},</nowiki>
estimate <nowiki>=> $options{estimate},</nowiki>
);
$tax_is_batch ||= $tax_engines{$_}->info->{batch};
}

Here we start the main loop over all packages. Skip any that are excluded by 'not_pkgpart', or by 'no_prepaid'. (freeside-daily sets this, as prepaid packages are billed by another mechanism.) And if there's an explicit 'pkg_list' then use only those.

foreach my $cust_pkg ( @{ $options{'pkg_list'} } ) {

next if $options{'not_pkgpart'}->{$cust_pkg->pkgpart};

my $part_pkg = $cust_pkg->part_pkg;

next if $options{'no_prepaid'} && $part_pkg->is_prepaid;

$log->debug('bill package '. $cust_pkg->pkgnum, %logopt);
warn " bill package ". $cust_pkg->pkgnum. "\n" if $DEBUG;

<nowiki>#? to avoid use of uninitialized value errors... ?</nowiki>
$cust_pkg->setfield('bill', '')
unless defined($cust_pkg->bill);

Call self_and_bill_linked() to find any billing add-on packages. Add-ons are additional package definitions (part_pkg records, plus all their accessories) that are linked to a “main” package definition, and generate additional charges on the bill. For example, a phone package that has a monthly fee plus long-distance usage billing would have an add-on voip_cdr package to do the usage billing.

Add-on packages always create separate cust_bill_pkg records, with the “pkgpart_override” field referencing the add-on package. Normally, these charges will show separately on the invoice, but they can be “bundled” with the main package by setting part_pkg_link.hidden. At this point we set a flag if this looks like a bundle package, as that will matter later.

my $real_pkgpart = $cust_pkg->pkgpart;
my %hash = $cust_pkg->hash;

<nowiki># we could implement this bit as FS::part_pkg::has_hidden, but we already</nowiki>
<nowiki># suffer from performance issues</nowiki>
$options{has_hidden} = 0;
my @part_pkg = $part_pkg->self_and_bill_linked;
$options{has_hidden} = 1 if ($part_pkg[1] && $part_pkg[1]->hidden);

== Create line items for each billable package. ==
As we create line items we'll push them onto $cust_bill_pkg{$pass} (either '' or 'no_auto'). First, balance transfers: if this package is being billed for the first time after a package change, and package balances are enabled, then credit the old package to zero out its balance, and make a line item to carry the balance forward to its successor package.

<nowiki># if this package was changed from another package,</nowiki>
<nowiki># and it hasn't been billed since then,</nowiki>
<nowiki># and package balances are enabled,</nowiki>
if ( $cust_pkg->change_pkgnum
and $cust_pkg->change_date >= ($cust_pkg->last_bill || 0)
and $cust_pkg->change_date < $invoice_time
and $conf->exists('pkg-balances') )
{
<nowiki># _transfer_balance will also create the appropriate credit</nowiki>
my @transfer_items = $self->_transfer_balance($cust_pkg);
<nowiki># $part_pkg[0] is the "real" part_pkg</nowiki>
my $pass = ($cust_pkg->no_auto || $part_pkg[0]->no_auto) ?
'no_auto' : '';
push @{ $cust_bill_pkg{$pass} }, @transfer_items;
<nowiki># treating this as recur, just because most charges are recur...</nowiki>
${$total_recur{$pass}} += $_->recur foreach @transfer_items;

<nowiki># currently not considering separate_bill here, as it's for </nowiki>
<nowiki># one-time charges only</nowiki>
}

Loop over the part_pkgs (first the “real” package definition, then the add-ons) and decide which invoice to put the charges on. Normally, $pass = '', which will put all charges on a single invoice, but no_auto packages have to go on another invoice, and each package with separate_bill needs yet another (so it will create entries in %total_setup, %total_recur, %cust_bill_pkg, etc.).

foreach my $part_pkg ( @part_pkg ) {

my $this_cust_pkg = $cust_pkg;
<nowiki># for add-on packages, copy the object to avoid leaking changes back to</nowiki>
<nowiki># the caller if pkg_list is in use; see RT#73607</nowiki>
if ( $part_pkg->get('pkgpart') != $real_pkgpart ) {
$this_cust_pkg = FS::cust_pkg->new({ %hash });
}

my $pass = '';
if ( $this_cust_pkg->separate_bill ) {
<nowiki># if no_auto is also set, that's fine. we just need to not have</nowiki>
<nowiki># invoices that are both auto and no_auto, and since the package</nowiki>
<nowiki># gets an invoice all to itself, it will only be one or the other.</nowiki>
$pass = $this_cust_pkg->pkgnum;
if (!exists $cust_bill_pkg{$pass}) { # it may not exist yet
push @passes, $pass;
$total_setup{$pass} = do { my $z = 0; \$z };
$total_recur{$pass} = do { my $z = 0; \$z };
<nowiki># it also needs its own tax context</nowiki>
$tax_engines{$pass} = FS::TaxEngine->new(
cust_main <nowiki>=> $self,</nowiki>
invoice_time => $invoice_time,
cancel <nowiki>=> $options{cancel},</nowiki>
estimate <nowiki>=> $options{estimate},</nowiki>
);
$cust_bill_pkg{$pass} = [];
}
} elsif ( ($this_cust_pkg->no_auto || $part_pkg->no_auto) ) {
$pass = 'no_auto';
}

The loop here checks whether the package is due for billing, and handles back-billing of multiple cycles. For example, if a monthly package's bill date is more than a month in the past, we'll bill as many months as necessary to bring the package up to date. The loop normally exits once the package bill date is in the future, but will exit after a single pass if we're billing on cancellation, or if freeside-daily -o is in use. It will also exit if there's an error or if the bill date isn't being incremented (as for a one-time charge).

_make_lines() is complex enough to have [#_make_lines its own section of the docs], but it generates a line item and does the following with it:

* appends it to the arrayref in 'line_items' (so, to $cust_bill_pkg{$pass}).
* adds its setup and recur fees to the running totals
* informs the tax engine about it (via the add_sale() method)
* increments the package's last-bill and next-bill dates
* returns an error if anything goes wrong

my $next_bill = $this_cust_pkg->getfield('bill') || 0;
my $error;
<nowiki># let this run once if this is the last bill upon cancellation</nowiki>
while ( $next_bill <= $cmp_time or $options{cancel} ) {
$error =
$self->_make_lines( 'part_pkg' <nowiki>=> $part_pkg,</nowiki>
'cust_pkg' <nowiki>=> $this_cust_pkg,</nowiki>
'precommit_hooks' <nowiki>=> \@precommit_hooks,</nowiki>
'line_items' <nowiki>=> $cust_bill_pkg{$pass},</nowiki>
'setup' <nowiki>=> $total_setup{$pass},</nowiki>
'recur' <nowiki>=> $total_recur{$pass},</nowiki>
'tax_engine' <nowiki>=> $tax_engines{$pass},</nowiki>
'time' <nowiki>=> $time,</nowiki>
'real_pkgpart' <nowiki>=> $real_pkgpart,</nowiki>
'options' <nowiki>=> \%options,</nowiki>
);

<nowiki># Stop if anything goes wrong</nowiki>
last if $error;

<nowiki># or if we're not incrementing the bill date.</nowiki>
last if ($this_cust_pkg->getfield('bill') || 0) == $next_bill;

<nowiki># or if we're letting it run only once</nowiki>
last if $options{cancel};

$next_bill = $this_cust_pkg->getfield('bill') || 0;

<nowiki>#stop if -o was passed to freeside-daily</nowiki>
last if $options{'one_recur'};
}
if ($error) {
$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return $error;
}

} #foreach my $part_pkg

} #foreach my $cust_pkg

== Identify applicable fees and tax adjustments and create line items. ==
At this point we have a list of package line items for each invoice the customer might receive. Now loop through the (possible) invoices and process non-package fees.

A “fee origin” (FS::FeeOrigin_Mixin) is an instruction or “order” to charge a fee<nowiki>; the classes are FS::cust_event_fee (fees generated by billing events, </nowiki>such as late fees) and FS::cust_pkg_reason_fee (unsuspension fees). When the fee is put on an invoice, the order is fulfilled; the billpkgnum of the fee line item is then recorded on the fee origin record.

The by_cust() method does a search for all fee origins for a customer; we limit it to records with billpkgnum = NULL to get fees that still need to be charged. Those are @pending_fees.

We process fees at this stage (after generating all package line items) because we need to know if the customer is going to receive an invoice or not. Fees can be defined (via the 'nextbill' flag) to be charged immediately, or on the customer's next bill. If all pending fees have 'nextbill' and there aren't any package charges on this invoice (@cust_bill_pkg is empty) then skip to the next invoice. If the customer has no package charges at all, then the 'nextbill' fees won't be charged at all on this billing date and will remain pending.

foreach my $pass (@passes) { # keys %cust_bill_pkg )

my @cust_bill_pkg = _omit_zero_value_bundles(@{ $cust_bill_pkg{$pass} });

warn "$me billing pass $pass\n"
<nowiki>#.Dumper(\@cust_bill_pkg)."\n"</nowiki>
if $DEBUG > 2;

<nowiki>###</nowiki>
<nowiki># process fees</nowiki>
<nowiki>###</nowiki>

my @pending_fees = FS::FeeOrigin_Mixin->by_cust($self->custnum,
hashref => { 'billpkgnum' => '' }
);
warn "$me found pending fees:\n".Dumper(\@pending_fees)."\n"
if @pending_fees and $DEBUG > 1;

<nowiki># determine whether to generate an invoice</nowiki>
my $generate_bill = scalar(@cust_bill_pkg) > 0;

foreach my $fee (@pending_fees) {
$generate_bill = 1 unless $fee->nextbill;
}

<nowiki># don't create an invoice with no line items, or where the only line </nowiki>
<nowiki># items are fees that are supposed to be held until the next invoice</nowiki>
next if !$generate_bill;

For each fee origin, do some things. Skip the fee if the fee definition is somehow inappropriate:

<nowiki># calculate fees...</nowiki>
my @fee_items;
foreach my $fee_origin (@pending_fees) {
my $part_fee = $fee_origin->part_fee;

<nowiki># check whether the fee is applicable before doing anything expensive:</nowiki>
<nowiki>#</nowiki>
<nowiki># if the fee def belongs to a different agent, don't charge the fee.</nowiki>
<nowiki># event conditions should prevent this, but just in case they don't,</nowiki>
<nowiki># skip the fee.</nowiki>
if ( $part_fee->agentnum and $part_fee->agentnum != $self->agentnum ) {
warn "tried to charge fee#".$part_fee->feepart .
" on customer#".$self->custnum." from a different agent.\n";
next;
}
<nowiki># also skip if it's disabled</nowiki>
next if $part_fee->disabled eq 'Y';

Then determine the “basis” for the fee, in case it's defined as a percentage of something. The fee origin may declare that the fee applies to a previous invoice (such as a finance charge). Otherwise, the fee applies to the ''current'' invoice. Since the current invoice doesn't exist yet, construct one temporarily that has the line items of the current invoice.

<nowiki># Decide which invoice to base the fee on.</nowiki>
my $cust_bill = $fee_origin->cust_bill;
if (!$cust_bill) {
<nowiki># Then link it to the current invoice. This isn't the real cust_bill</nowiki>
<nowiki># object that will be inserted--in particular there are no taxes yet.</nowiki>
<nowiki># If you want to charge a fee on the total invoice amount including</nowiki>
<nowiki># taxes, you have to put the fee on the next invoice.</nowiki>
$cust_bill = FS::cust_bill->new({
'custnum' <nowiki>=> $s</nowiki>elf->custnum,
'cust_bill_pkg' => \@cust_bill_pkg,
'charged' <nowiki>=> ${ $total_setup{$pass} } +</nowiki>
${ $total_recur{$pass} },
});
<nowiki># If the origin is for a specific package, then only apply the fee to</nowiki>
<nowiki># line items from that package.</nowiki>
if ( my $cust_pkg = $fee_origin->cust_pkg ) {
my @charge_fee_on_item;
my $charge_fee_on_amount = 0;
foreach (@cust_bill_pkg) {
if ($_->pkgnum == $cust_pkg->pkgnum) {
push @charge_fee_on_item, $_;
$charge_fee_on_amount += $_->setup + $_->recur;
}
}
$cust_bill->set('cust_bill_pkg', \@charge_fee_on_item);
$cust_bill->set('charged', $charge_fee_on_amount);
}

} # $cust_bill is now set

FS::part_fee->lineitem() asks the fee definition to make a cust_bill_pkg record for this fee, as applied to a specific invoice. After the invoice is inserted, we're going to need to record the billpkgnum on the fee origin record. FS::cust_bill_pkg->insert knows to do this.

Then append any fee line items to the invoice, add them to the running totals, and send them to the tax engine.

<nowiki># calculate the fee</nowiki>
my $fee_item = $part_fee->lineitem($cust_bill) or next;
<nowiki># link this so that we can clear the marker on inserting the line item</nowiki>
$fee_item->set('fee_origin', $fee_origin);
push @fee_items, $fee_item;

}

<nowiki># add fees to the invoice</nowiki>
foreach my $fee_item (@fee_items) {

push @cust_bill_pkg, $fee_item;
${ $total_setup{$pass} } += $fee_item->setup;
${ $total_recur{$pass} } += $fee_item->recur;

my $part_fee = $fee_item->part_fee;
my $fee_location = $self->ship_location; # I think?

my $error = $tax_engines{''}->add_sale($fee_item);

return $error if $error;

}

Add the special “postal invoice fee”. This was a per-invoice flat fee for sending customers a printed invoice; like all “old” fees it worked by adding a one-time charge. charge_postal_fee() is the method to do that. It adds a one-time package to the customer ''in the middle of billing'', after all their other packages have been billed, for exactly the same reason that other fees are processed at this point in billing. This feature is obsolete so I won't say any more.

<nowiki># XXX implementation of fees is supposed to make this go away...</nowiki>
if ( scalar( grep { $_->recur && $_->recur > 0 } @cust_bill_pkg) ||
!$conf->exists('postal_invoice-recurring_only')
)
{

my $postal_pkg = $self->charge_postal_fee();
if ( $postal_pkg && !ref( $postal_pkg ) ) {

$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return "can't charge postal invoice fee for customer ".
$self->custnum. ": $postal_pkg";

} elsif ( $postal_pkg ) {

my $real_pkgpart = $postal_pkg->pkgpart;
<nowiki># we could implement this bit as FS::part_pkg::has_hidden, but we already</nowiki>
<nowiki># suffer from performance issues</nowiki>
$options{has_hidden} = 0;
my @part_pkg = $postal_pkg->part_pkg->self_and_bill_linked;
$options{has_hidden} = 1 if ($part_pkg[1] && $part_pkg[1]->hidden);

foreach my $part_pkg ( @part_pkg ) {
my %postal_options = %options;
delete $postal_options{cancel};
my $error =
$self->_make_lines( 'part_pkg' <nowiki>=> $part_pkg,</nowiki>
'cust_pkg' <nowiki>=> $postal_pkg,</nowiki>
'precommit_hooks' <nowiki>=> \@precommit_hooks,</nowiki>
'line_items' <nowiki>=> \@cust_bill_pkg,</nowiki>
'setup' <nowiki>=> $total_setup{$pass},</nowiki>
'recur' <nowiki>=> $total_recur{$pass},</nowiki>
'tax_engine' <nowiki>=> $tax_engines{$pass},</nowiki>
'time' <nowiki>=> $time,</nowiki>
'real_pkgpart' <nowiki>=> $real_pkgpart,</nowiki>
'options' <nowiki>=> \%postal_options,</nowiki>
);
if ($error) {
$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return $error;
}
}

<nowiki># it's silly to have a zero value postal_pkg, but....</nowiki>
@cust_bill_pkg = _omit_zero_value_bundles(@cust_bill_pkg);

}

}

Tax adjustments (FS::cust_tax_adjustment) are manual adjustment records that work much like fee origins, but are created manually. If a tax adjustment exists for the customer, and doesn't have a billpkgnum yet, and an invoice is being generated, then create a line item with the amount and itemdesc specified in the tax adjustment. Note that this line item ''is a tax line item'': it has pkgnum = 0, feepart = NULL, and recur = 0. On that line item, set 'cust_tax_adjustment' to the tax adjustment record so that cust_bill_pkg->insert can record that the adjustment was billed.

<nowiki>#add tax adjustments</nowiki>
<nowiki>#XXX does this work with batch tax engines?</nowiki>
warn "adding tax adjustments...\n" if $DEBUG > 2;
foreach my $cust_tax_adjustment (
qsearch('cust_tax_adjustment', { 'custnum' <nowiki>=> $self->custnum,</nowiki>
'billpkgnum' => '',
}
)
) {

my $tax = sprintf('%.2f', $cust_tax_adjustment->amount );

my $itemdesc = $cust_tax_adjustment->taxname;
$itemdesc = '' if $itemdesc eq 'Tax';

push @cust_bill_pkg, new FS::cust_bill_pkg {
'pkgnum' <nowiki>=> 0,</nowiki>
'setup' <nowiki>=> $tax,</nowiki>
'recur' <nowiki>=> 0,</nowiki>
'sdate' <nowiki>=> '',</nowiki>
'edate' <nowiki>=> '',</nowiki>
'itemdesc' <nowiki>=> $itemdesc,</nowiki>
'itemcomment' => $cust_tax_adjustment->comment,
'cust_tax_adjustment' => $cust_tax_adjustment,
<nowiki>#'cust_bill_pkg_tax_location' => \@cust_bill_pkg_tax_location,</nowiki>
};

}

== Create the invoice record. ==
Before storing the record, calculate some totals:

* cust_bill.charged: the sum of charges on this invoice (taxes will be added later)
* billing_balance: the customer's balance before this invoice.
* previous_balance: the customer's balance immediately after their previous invoice, determined as (billing_balance + charged) on that invoice.

Note that ''billing_balance and previous_balance are obsolete''. They were historically used to create invoice summary sections (“Balance on previous invoice”, “Payments/credits since then”), but since 2014 we actually look at the transaction history. This method produces correct results if transactions get voided.

my $charged = sprintf('%.2f', ${ $total_setup{$pass} } + ${ $total_recur{$pass} } );

my $balance = $self->balance;

my $previous_bill = qsearchs({ 'table' <nowiki>=> 'cust_bill',</nowiki>
'hashref' <nowiki>=> { custnum=>$self->custnum },</nowiki>
'extra_sql' => 'ORDER BY _date DESC LIMIT 1',
});
my $previous_balance =
$previous_bill
? ( $previous_bill->billing_balance + $previous_bill->charged )
: 0;

Now store the record. FS::cust_bill::insert() will notice the arrayref of line items and insert all of them. Line items with 'fee_origin' or 'tax_adjustment' references will also trigger updates to those records.

The 'no_commit' option is for simulating billing, and will stop the record from being inserted. Currently this is used only with term prepayment discounts (which are no longer a supported feature) to figure out the “amount saved” over the prepaid term.

Note that (like almost anywhere else in Freeside) $FS::UID::AutoCommit = 0 will also prevent bill() from committing its changes. This is how quotations work: FS::quotation->estimate opens a transaction, inserts packages, runs bill(), and then reverts the whole transaction.

$log->debug('creating the new invoice', %logopt);
warn "creating the new invoice\n" if $DEBUG;
<nowiki>#create the new invoice</nowiki>
my $cust_bill = new FS::cust_bill ( {
'custnum' <nowiki>=> $self->custnum,</nowiki>
'_date' <nowiki>=> $invoice_time,</nowiki>
'charged' <nowiki>=> $charged,</nowiki>
'billing_balance' <nowiki>=> $balance,</nowiki>
'previous_balance' <nowiki>=> $previous_balance,</nowiki>
'invoice_terms' <nowiki>=> $options{'invoice_terms'},</nowiki>
'cust_bill_pkg' <nowiki>=> \@cust_bill_pkg,</nowiki>
'pending' <nowiki>=> 'Y', # clear this after doing taxes</nowiki>
} );

if (!$options{no_commit}) {
<nowiki># probably we ought to insert it as pending, and then rollback</nowiki>
<nowiki># without ever un-pending it</nowiki>
$error = $cust_bill->insert;
if ( $error ) {
$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return "can't create invoice for customer #". $self->custnum. ": $error";
}

}

== Calculate taxes and insert tax line items. ==
$tax_is_batch was previously set to true if we're using a batch tax processor. In that case, don't do any of this; just leave the invoice flagged as pending and FS::Cron::tax_batch will do the rest.

Otherwise: get the tax engine we created for this invoice pass, and call calculate_taxes(), passing the pending invoice. This will return an arrayref of line items, or throw an exception. If it throws an exception, rollback and exit.

<nowiki># calculate and append taxes</nowiki>
if ( ! $tax_is_batch) {
local $@;
my $arrayref = eval { $tax_engines{$pass}->calculate_taxes($cust_bill) };

if ( $@ ) {
$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return $@;
}

Append each tax line item to the invoice (directly, by setting invnum and inserting them) and add the charges to a running total. Tax line items have 'setup' but not 'recur' charges.

<nowiki># or should this be in TaxEngine?</nowiki>
my $total_tax = 0;
foreach my $taxline ( @$arrayref ) {
$total_tax += $taxline->setup;
$taxline->set('invnum' => $cust_bill->invnum); # just to be sure
push @cust_bill_pkg, $taxline; # for return_bill

if (!$options{no_commit}) {
my $error = $taxline->insert;
if ( $error ) {
$dbh->rollback if $oldAutoCommit;
return $error;
}
}

}

Add the total tax to the total invoice charges, remove the pending flag, and then update the invoice record. Then we're done.

<nowiki># add tax to the invoice amount and finalize it</nowiki>
${ $total_setup{$pass} } = sprintf('%.2f', ${ $total_setup{$pass} } + $total_tax);
$charged = sprintf('%.2f', $charged + $total_tax);
$cust_bill->set('charged', $charged);
$cust_bill->set('pending', '');

if (!$options{no_commit}) {
my $error = $cust_bill->replace;
if ( $error ) {
$dbh->rollback if $oldAutoCommit;
return $error;
}
}

} # if !$tax_is_batch
<nowiki># if it IS batch, then we'll do all this in process_tax_batch</nowiki>

After dealing with this invoice, push it to the 'return_bill' arrayref if there is one. This will allow the caller to see which invoices were created if they want.

push @{$options{return_bill}}, $cust_bill if $options{return_bill};

} #foreach my $pass ( keys %cust_bill_pkg )

Run any post-invoicing pre-commit hooks. These are currently used for only one thing: for packages with time-limited discounts, we need to increment the number of discount months used after completely billing the package, but before doing anything else. FS::part_pkg::discount_Mixin sets this up.

Then commit changes if needed, and return.

foreach my $hook ( @precommit_hooks ) {
eval {
&{$hook}; #($self) ?
} unless $options{no_commit};
if ( $@ ) {
$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return "$@ running precommit hook $hook\n";
}
}

$dbh->commit or die $dbh->errstr if $oldAutoCommit && !$options{no_commit};

''; #no error
}

= FS::cust_main::Billing::_make_lines =
This is the other most important billing method. It takes a single package and package definition, and creates at most one line item for that package. This is called only from bill() and should probably never be used any other way, since it updates the package's billing dates.

== Gather parameters ==
$cust_pkg is the package. $part_pkg is the package definition, either the “real” one for this package or a billing add-on. We temporarily set the cust_pkg->pkgpart property to match it so that we can call cust_pkg methods that delegate to “$self->part_pkg”. We then copy all the cust_pkg's fields into %hash.

$time is the “billing as of” date. %options is all the options that were passed to bill().

The other $param{} elements are accumulators passed from outside so that this method can add stuff to them.

$cust_location here isn't used for anything, because of tax engine refactoring.

The reference to freq_override is one last attempt to make sure term discounts aren't set up in a radically absurd way.

$lineitems is for remembering if we've done anything that requires creating a line item; if not then we'll exit without pushing one to the array.

@details will be used for any detail records that we attach to this line item. Those are additional lines of text shown below the line item on the invoice.

$cmp_time, as in bill(), is either “right now” or “the end of today” depending on the '''next-bill-ignore-time''' option.

sub _make_lines {
my ($self, %params) = @_;

local($DEBUG) = $FS::cust_main::DEBUG if $FS::cust_main::DEBUG > $DEBUG;

my $part_pkg = $params{part_pkg} or die "no part_pkg specified";
my $cust_pkg = $params{cust_pkg} or die "no cust_pkg specified";
my $cust_location = $cust_pkg->tax_location;
my $precommit_hooks = $params{precommit_hooks} or die "no precommit_hooks specified";
my $cust_bill_pkgs = $params{line_items} or die "no line buffer specified";
my $total_setup = $params{setup} or die "no setup accumulator specified";
my $total_recur = $params{recur} or die "no recur accumulator specified";
my $time = $params{'time'} or die "no time specified";
my (%options) = %{$params{options}};

my $tax_engine = $params{tax_engine};

if ( $part_pkg->freq ne '1' and ($options{'freq_override'} || 0) > 0 ) {
<nowiki># this should never happen</nowiki>
die 'freq_override billing attempted on non-monthly package '.
$cust_pkg->pkgnum;
}

my $dbh = dbh;
my $real_pkgpart = $params{real_pkgpart};
my %hash = $cust_pkg->hash;
my $old_cust_pkg = new FS::cust_pkg \%hash;

my @details = ();
my $lineitems = 0;

$cust_pkg->pkgpart($part_pkg->pkgpart);

my $cmp_time = ( $conf->exists('next-bill-ignore-time')
? day_end( $time )
: $time
);

== Charge setup fee and set setup date ==
Setup fees are charged the first time a package is billed. %setup_param will be passed to the calc_setup() method; because of how discounts work, it needs to know if this is an add-on package, and to report if it's applying any discounts. $setup and $unitsetup will become those fields on the cust_bill_pkg record.

The conditions for charging the setup fee are exactly as described in the inline comment. Note that a package that's canceled before first billing will never be charged a setup fee (or any other fee). We assume the package was ordered by mistake.

Note also that if the package has an expire date <= $cmp_time then it won't be charged. Normally the package would be canceled already. This check is for the case where the package isn't expired yet (in real time) but is being billed for a date after it will expire. For example, if a package has a next bill date of Feb 1, but an expiration date of Jan 31, and you're running freeside-daily -y 3 to print invoices 3 days in advance, then billing will trigger for the customer on Jan 28. The package won't be canceled yet. However, we should ''not'' preprint an invoice for this package because it ''will'' expire before its real billing date arrives. This is a real case that has come up.

Possibly we should check adjourn dates also, since a suspended package is not supposed to get set up.

<nowiki>###</nowiki>
<nowiki># bill setup</nowiki>
<nowiki>###</nowiki>

my $setup = 0;
my $unitsetup = 0;
my @setup_discounts = ();
my %setup_param = ( 'discounts' <nowiki>=> \@setup_discounts,</nowiki>
'real_pkgpart' <nowiki>=> $params{real_pkgpart}</nowiki>
);
my $setup_billed_currency = '';
my $setup_billed_amount = 0;
<nowiki># Conditions for setting setup date and charging the setup fee:</nowiki>
<nowiki># - this is not a recurring-only billing run</nowiki>
<nowiki># - and the package is not currently being canceled</nowiki>
<nowiki># - and, unless we're specifically told otherwise via 'resetup':</nowiki>
<nowiki># </nowiki> - it doesn't already HAVE a setup date
<nowiki># </nowiki> - or a start date in the future
<nowiki># </nowiki> - and it's not suspended
<nowiki># - and it doesn't have an expire date in the past</nowiki>
<nowiki>#</nowiki>
<nowiki># The "disable_setup_suspended" option is now obsolete; we never set the</nowiki>
<nowiki># setup date on a suspended package.</nowiki>
if ( ! $options{recurring_only}
and ! $options{cancel}
and ( $options{'resetup'}
|| ( ! $cust_pkg->setup
&& ( ! $cust_pkg->start_date
|| $cust_pkg->start_date <= $cmp_time
)
&& ( ! $cust_pkg->getfield('susp') )
)
)
and ( ! $cust_pkg->expire
|| $cust_pkg->expire > $cmp_time )
)
{

warn " bill setup\n" if $DEBUG > 1;

If 'waive_setup' is set on the package then skip this step.

Calculate the setup fee. FS::cust_pkg->calc_setup just delegates to its part_pkg (which we've overridden if we're processing an add-on package right now). The following part_pkg classes have a calc_setup method:

* flat and recur_Common: Calls prorate_setup(), about which [#Deferred prorate see below]. Adds any “additional_info” strings to @details (this is used for putting notes on one-time charges). Calls base_setup() to get the setup fee from the package definition, then calc_discount() to apply any discounts, then multiplies the result by quantity. Almost all packages use this logic.
* currency_fixed: Calls prorate_setup(), then base_setup() (which does the currency conversion). Oddly, it doesn't multiply by quantity; this is probably a bug, since it does multiply by quantity in calc_recur().
* delayed_Mixin: If the 'delay_setup' option is off, pushes back the package bill date by some number of days so that the start of recurring billing is delayed. Then calls whatever calc_setup() method would otherwise apply.
* rt_field and (the obsolete) voip_sqlradacct: Returns the 'setup_fee' package option, without multiplying by quantity or applying discounts.

Also calculate the per-unit setup fee (for displaying on the invoice). The base_setup() method, by definition, returns the non-discounted per-unit setup fee.

Currency packages also return (via $setup_param) the currency and amount before doing conversion; we remember those here and will put them on the invoice later.

unless ( $cust_pkg->waive_setup ) {
$lineitems++;

$setup = eval { $cust_pkg->calc_setup( $time, \@details, \%setup_param ) };
return "$@ running calc_setup for $cust_pkg\n"
if $@;

<nowiki># Only increment unitsetup here if there IS a setup fee.</nowiki>
<nowiki># prorate_defer_bill may cause calc_setup on a setup-stage package</nowiki>
<nowiki># to return zero, and the setup fee to be charged later. (This happens</nowiki>
<nowiki># when it's first billed on the prorate cutoff day. RT#31276.)</nowiki>
if ( $setup ) {
$unitsetup = $cust_pkg->base_setup()
|| $setup; #XXX uuh
}

if ( $setup_param{'billed_currency'} ) {
$setup_billed_currency = delete $setup_param{'billed_currency'};
$setup_billed_amount <nowiki>= delete $setup_param{'billed_amount'};</nowiki>
}
}

Assign the setup date. The package may already have a setup date (because of 'resetup'). Otherwise, if it has a designated start date, use that (and clear the start date). Otherwise, it started billing now so set it to now.

if ( $cust_pkg->get('setup') ) {
<nowiki># don't change it</nowiki>
} elsif ( $cust_pkg->get('start_date') ) {
<nowiki># this allows start_date to be used to set the first bill date</nowiki>
$cust_pkg->set('setup', $cust_pkg->get('start_date'));
} else {
<nowiki># if unspecified, start it right now</nowiki>
$cust_pkg->set('setup', $time);
}

$cust_pkg->setfield('start_date', '')
if $cust_pkg->start_date;

}

=== Deferred prorate ===
This is an optional mode for prorate packages (and flat monthly packages using sync_bill_date), enabled by the prorate_defer_bill option. Rather than charging a fractional month at setup time, the package is billed on the first cutoff day for a full month ''plus'' the fractional month preceding the cutoff day.

In packages that are eligible for deferred prorate, calc_setup() calls the calc_prorate() method, which checks whether the flag is present. If so, calc_prorate sets the setup date (to now) and the next bill date (to the upcoming cutoff day) but doesn't set last_bill because the package hasn't really been billed. Then it returns a true value, which tells calc_setup ''not'' to charge a setup fee. The package will then not be charged a recurring fee either, because the next bill date is in the future.

(If the setup date is itself a cutoff day, then calc_prorate will ignore the flag and bill the package for a full month as usual. It is highly recommended that you use the prorate_round_day option in combination with this so that the “full month” really is a full month.)

When the first recurring billing date arrives, calc_recur() notices that there's a bill date but no last_bill, and charges for the partial month in the past and full month in the future. (The start and end dates on the line item will reflect this.) It will also call calc_setup() and apply any setup fee defined by the package.

== Charge the recurring fee and adjust the bill date ==
The comment here explains the logic for whether to charge a recurring fee.

At this point, if it still has a start_date, then either the start_date is in the future or the package couldn't start billing for some other reason (like that it's on hold).

See the note above about the expire date; this part is slightly different in that if the package actually ''has'' expired and we're billing it on cancellation, then charging a recurring fee is allowed.

<nowiki>###</nowiki>
<nowiki># bill recurring fee</nowiki>
<nowiki>### </nowiki>

my $recur = 0;
my $unitrecur = 0;
my @recur_discounts = ();
my $recur_billed_currency = '';
my $recur_billed_amount = 0;
my $sdate;

my $override_quantity;

<nowiki># Conditions for billing the recurring fee:</nowiki>
<nowiki># - the package doesn't have a future start date</nowiki>
<nowiki># - and it's not suspended</nowiki>
<nowiki># </nowiki> - unless suspend_bill is enabled on the package or package def
<nowiki># </nowiki> - but still not, if the package is on hold
<nowiki># </nowiki> - or it's suspended for a delayed cancellation
<nowiki># - and its next bill date is in the past</nowiki>
<nowiki># </nowiki> - or it doesn't have a next bill date yet
<nowiki># </nowiki> - or it's a one-time charge
<nowiki># </nowiki> - or it's a CDR plan with the "bill_every_call" option
<nowiki># </nowiki> - or it's being canceled
<nowiki># - and it doesn't have an expire date in the past (this can happen with</nowiki>
<nowiki># </nowiki> advance billing)
<nowiki># </nowiki> - again, unless it's being canceled
if ( ! $cust_pkg->start_date
and
( ! $cust_pkg->susp
|| ( $cust_pkg->susp != $cust_pkg->order_date
&& ( $cust_pkg->option('suspend_bill',1)
|| ( $part_pkg->option('suspend_bill', 1)
&& ! $cust_pkg->option('no_suspend_bill',1)
)
)
)
|| $cust_pkg->is_status_delay_cancel
)
and
( $part_pkg->freq ne '0' && ( $cust_pkg->bill || 0 ) <= $cmp_time )
|| ( $part_pkg->plan eq 'voip_cdr'
&& $part_pkg->option('bill_every_call')
)
|| $options{cancel}

and
( ! $cust_pkg->expire
|| $cust_pkg->expire > $cmp_time
|| $options{cancel}
)
) {

Reset the package's usage cap if it has one:

* For voip_cdr, this is for “usage pools”. It clears all cdr_cust_pkg_usage records from the package and resets the cust_pkg_usage's minutes remaining.
* For flat (and other flat-like packages), this finds svc_accts attached to the package and resets their byte usage counters. This is for RADIUS services with data caps.

<nowiki># XXX should this be a package event? </nowiki> probably. events are called
<nowiki># at collection time at the moment, though...</nowiki>
$part_pkg->reset_usage($cust_pkg, 'debug'=>$DEBUG)
if $part_pkg->can('reset_usage') && !$options{'no_usage_reset'};
<nowiki>#don't want to reset usage just cause we want a line item??</nowiki>
<nowiki>#&& $part_pkg->pkgpart == $real_pkgpart;</nowiki>

At this point we've decided to charge a recurring fee, so increment $lineitems.

“$sdate” is the effective billing date. It represents the end of the previous cycle and the start of the next. The next bill date will be set to $sdate + one billing period.

For a new package, $sdate is the setup date (so, the start date if there was one); if it's been billed before then it's the billing date.

Prorate math will normally charge for the period from $sdate to the next prorate cycle date after that. After that, it will set the next bill date to the prorate cycle date, so that future billing cyles run from one prorate cycle date to the next. The result is that the customer is charged for a partial month, then for full months after that.

$increment_next_bill is, yes, a flag saying to increment the bill date. We won't increment for one-time charges, or packages that are being canceled. The check for the bill date <= $cmp_time is now redundant, since we won't bill the package at all.

warn " bill recur\n" if $DEBUG > 1;
$lineitems++;

<nowiki># XXX shared with $recur_prog</nowiki>
$sdate = ( $options{cancel} ? $cust_pkg->last_bill : $cust_pkg->bill )
|| $cust_pkg->setup
|| $time;

<nowiki>#over two params! </nowiki> lets at least switch to a hashref for the rest...
my $increment_next_bill = ( $part_pkg->freq ne '0'
&& ( $cust_pkg->getfield('bill') || 0 ) <= $cmp_time
&& !$options{cancel}
);

The most important part of the entire module: calculate the recurring fee. Call FS::part_pkg::calc_recur (or calc_cancel, if we're canceling the package).

This is ''not a well-behaved interface''. The arguments are all passed by reference (they're used to pass values back out), and they include the contents of %setup_param, so packages can use the %param hashref argument to pass notes between the calc_setup and calc_recur stages. Discounts, especially, use this.

* $sdate is the effective bill date, @details is the array of line item details, and %param contains everything else.
* 'discounts' is an arrayref of objects that will be inserted with the line item to track which discounts applied.
* 'precommit_hooks' and 'real_pkgpart' are used to keep track of the months remaining on term-limited discounts: if real_pkgpart matches the pkgpart of the package, then it's not an add-on package, so a month (or other amount) should be deducted. In that case a callback will be appended to precommit_hooks to do that at the end.
* 'freq_override' should be ignored.
* 'setup_fee' allows calc_recur to charge a setup fee. This seems bizarre but is needed for certain prorate cases.
* 'increment_next_bill' is checked in recur_Common to decide whether to apply the fixed recurring charge for voip_cdr and other usage-based packages. The intent seems to be to not bill those charges on cancellation.

my %param = ( %setup_param,
'precommit_hooks' <nowiki>=> $precommit_hooks,</nowiki>
'increment_next_bill' => $increment_next_bill,
'discounts' <nowiki>=> \@recur_discounts,</nowiki>
'real_pkgpart' <nowiki>=> $real_pkgpart,</nowiki>
'freq_override' => $options{freq_override} || '',
'setup_fee' <nowiki>=> 0,</nowiki>
);

my $method = $options{cancel} ? 'calc_cancel' : 'calc_recur';

<nowiki># There may be some part_pkg for which this is wrong. </nowiki> Only those
<nowiki># which can_discount are supported.</nowiki>
<nowiki># (the UI should prevent adding discounts to these at the moment)</nowiki>

warn "calling $method on cust_pkg ". $cust_pkg->pkgnum.
" for pkgpart ". $cust_pkg->pkgpart.
" with params ". join(' / ', map "$_=>$param{$_}", keys %param). "\n"
if $DEBUG > 2;

$recur = eval { $cust_pkg->$method( \$sdate, \@details, \%param ) };
return "$@ running $method for $cust_pkg\n"
if ( $@ );

Handle a special case where calc_cancel() finds that the package isn't eligible to be billed on cancellation. This affects the decision about whether to create a line item at all.

if ($recur eq 'NOTHING') {
<nowiki># then calc_cancel (or calc_recur but that's not used) has declined to</nowiki>
<nowiki># generate a recurring lineitem at all. treat this as zero, but also </nowiki>
<nowiki># try not to generate a lineitem.</nowiki>
$recur = 0;
$lineitems--;
}

Process more things returned by calc_recur. Ask the package for the unit recurring charge (passing $sdate because packages with an intro rate will report a different amount in the intro period).

If calc_recur did a currency conversion, record how it happened.

If calc_recur reported a quantity, record that quantity and adjust unitrecur. sql_external packages can do this.

<nowiki>#base_cancel???</nowiki>
$unitrecur = $cust_pkg->base_recur( \$sdate ) || $recur; #XXX uuh, better

if ( $param{'billed_currency'} ) {
$recur_billed_currency = delete $param{'billed_currency'};
$recur_billed_amount <nowiki>= delete $param{'billed_amount'};</nowiki>
}

if ( $param{'override_quantity'} ) {
$override_quantity = $param{'override_quantity'};
$unitrecur = $recur / $override_quantity;
}

The other most important part of the module: increment the next bill date so that the package cycles.

As always in Freeside, adding some period to a bill date is done by the FS::part_pkg->add_freq method.

Deal with the case where this is a supplemental package. A supplemental package is “geared” to its main package: there's a fixed frequency ratio between them so that every N cycles they bill on the same day. If it's an integer multiple then that's easy (just add the main package period that man times). Otherwise, check whether the next bill is when they're due to sync up, and if so, sync them. This uses a rather crude “they're within 15 days” heuristic.

Otherwise, just call add_freq() using this package's frequency. Note that if the package is being prorated, $sdate will already have been moved to a prorate cycle date so that adding one month yields the right result.

if ( $increment_next_bill ) {

my $next_bill;

if ( my $main_pkg = $cust_pkg->main_pkg ) {
<nowiki># supplemental package</nowiki>
<nowiki># to keep in sync with the main package, simulate billing at </nowiki>
<nowiki># its frequency</nowiki>
my $main_pkg_freq = $main_pkg->part_pkg->freq;
my $supp_pkg_freq = $part_pkg->freq;
if ( $supp_pkg_freq == 0 or $main_pkg_freq == 0 ) {
<nowiki># the UI should prevent setting up packages like this, but just</nowiki>
<nowiki># in case</nowiki>
return "unable to calculate supplemental package period ratio";
}
my $ratio = $supp_pkg_freq / $main_pkg_freq;
if ( $ratio == int($ratio) ) {
<nowiki># simple case: main package is X months, supp package is X*A months,</nowiki>
<nowiki># advance supp package to where the main package will be in A cycles.</nowiki>
$next_bill = $sdate;
for (1..$ratio) {
$next_bill = $part_pkg->add_freq( $next_bill, $main_pkg_freq );
}
} else {
<nowiki># harder case: main package is X months, supp package is Y months.</nowiki>
<nowiki># advance supp package by Y months. then if they're within half a </nowiki>
<nowiki># month of each other, resync them. this may result in the period</nowiki>
<nowiki># not being exactly Y months.</nowiki>
$next_bill = $part_pkg->add_freq( $sdate, $supp_pkg_freq );
my $main_next_bill = $main_pkg->bill;
if ( $main_pkg->bill <= $time ) {
<nowiki># then the main package has not yet been billed on this cycle;</nowiki>
<nowiki># predict what its bill date will be.</nowiki>
$main_next_bill =
$part_pkg->add_freq( $main_next_bill, $main_pkg_freq );
}
if ( abs($main_next_bill - $next_bill) < 86400*15 ) {
$next_bill = $main_next_bill;
}
}

} else {
<nowiki># the normal case, not a supplemental package</nowiki>
$next_bill = $part_pkg->add_freq($sdate, $options{freq_override} || 0);
return "unparsable frequency: ".
($options{freq_override} || $part_pkg->freq)
if $next_bill == -1;
}

Set the “last bill” date to either the previous value of “next bill” or the setup date.

Also, if there's any second-pass setup fee, apply that to the line item.

'discount_left_setup' is a register used by the discount code for any flat-amount discount that's available for applying to the setup fee. Apply it here. This probably should set up a cust_bill_pkg_discount record but it's such an obscure case that we haven't needed it yet.

<nowiki>#pro-rating magic - if $recur_prog fiddled $sdate, want to use that</nowiki>
<nowiki># only for figuring next bill date, nothing else, so, reset $sdate again</nowiki>
<nowiki># here</nowiki>
$sdate = $cust_pkg->bill || $cust_pkg->setup || $time;
<nowiki>#no need, its in $hash{last_bill}# my $last_bill = $cust_pkg->last_bill;</nowiki>
$cust_pkg->last_bill($sdate);

$cust_pkg->setfield('bill', $next_bill );

}

if ( $param{'setup_fee'} ) {
<nowiki># Add an additional setup fee at the billing stage.</nowiki>
<nowiki># Used for prorate_defer_bill.</nowiki>
$setup += $param{'setup_fee'};
$unitsetup = $cust_pkg->base_setup();
$lineitems++;
}

if ( defined $param{'discount_left_setup'} ) {
foreach my $discount_setup ( values %{$param{'discount_left_setup'}} ) {
$setup -= $discount_setup;
}
}
} # end of recurring fee

warn "\$setup is undefined" unless defined($setup);
warn "\$recur is undefined" unless defined($recur);
warn "\$cust_pkg->bill is undefined" unless defined($cust_pkg->bill);

== Construct a cust_bill_pkg record ==
If $lineitems is zero then billing was entirely skipped and there's nothing to do.

Otherwise, save the cust_pkg record with the new bill and last_bill dates. Only do that when processing the base package; add-on packages don't increment the bill date.

Clean up $setup and $recur. The 'allow_negative_charges' config is no longer used.

<nowiki>###</nowiki>
<nowiki># If there's line items, create em cust_bill_pkg records</nowiki>
<nowiki># If $cust_pkg has been modified, update it (if we're a real pkgpart)</nowiki>
<nowiki>###</nowiki>

if ( $lineitems ) {

if ( $cust_pkg->modified && $cust_pkg->pkgpart == $real_pkgpart ) {
<nowiki># hmm.. and if just the options are modified in some weird price plan?</nowiki>

warn " package ". $cust_pkg->pkgnum. " modified; updating\n"
if $DEBUG >1;

my $error = $cust_pkg->replace( $old_cust_pkg,
'depend_jobnum'=>$options{depend_jobnum},
'options' => { $cust_pkg->options },
)
unless $options{no_commit};
return "Error modifying pkgnum ". $cust_pkg->pkgnum. ": $error"
if $error; #just in case
}

$setup = sprintf( "%.2f", $setup );
$recur = sprintf( "%.2f", $recur );
if ( $setup < 0 && ! $conf->exists('allow_negative_charges') ) {
return "negative setup $setup for pkgnum ". $cust_pkg->pkgnum;
}
if ( $recur < 0 && ! $conf->exists('allow_negative_charges') ) {
return "negative recur $recur for pkgnum ". $cust_pkg->pkgnum;
}

One more check. If the setup and recurring fees are both zero, we might skip creating a line item. It depends. Create a zero-dollar line item if:

* The setup or recur is zero ''because it was discounted to zero'', and the '''discount-show-always''' config is on. The invoice will then show the applied discount.
* This is the main package in a ''bundle''. Its add-on packages will add hidden line items for the charges, but we need this one so that the bundle is shown with the correct package name.
* Either the cust_pkg or the part_pkg has the 'setup_show_zero' or 'recur_show_zero' flag. (It's not clear to me why these are separate flags, since they have exactly the same effect.)

my $discount_show_always = $conf->exists('discount-show-always')
&& ( ($setup == 0 && scalar(@setup_discounts))
|| ($recur == 0 && scalar(@recur_discounts))
);

if ( $setup != 0
|| $recur != 0
|| (!$part_pkg->hidden && $options{has_hidden}) #include some $0 lines
|| $discount_show_always
|| ($setup == 0 && $cust_pkg->_X_show_zero('setup'))
|| ($recur == 0 && $cust_pkg->_X_show_zero('recur'))
)
{

warn " charges (setup=$setup, recur=$recur); adding line items\n"
if $DEBUG > 1;

Find any package details that are set to display as “invoice details” and add them to the list.

Then actually create the cust_bill_pkg. Huzzah!

my @cust_pkg_detail = map { $_->detail } $cust_pkg->cust_pkg_detail('I');
if ( $DEBUG > 1 ) {
warn " adding customer package invoice detail: $_\n"
foreach @cust_pkg_detail;
}
push @details, @cust_pkg_detail;
my $cust_bill_pkg = new FS::cust_bill_pkg {
'pkgnum' <nowiki>=> $cust_pkg->pkgnum,</nowiki>
'setup' <nowiki>=> $setup,</nowiki>
'unitsetup' <nowiki>=> sprintf('%.2f', $unitsetup),</nowiki>
'setup_billed_currency' => $setup_billed_currency,
'setup_billed_amount' <nowiki>=> $setup_billed_amount,</nowiki>
'recur' <nowiki>=> $recur,</nowiki>
'unitrecur' <nowiki>=> sprintf('%.2f', $unitrecur),</nowiki>
'recur_billed_currency' => $recur_billed_currency,
'recur_billed_amount' <nowiki>=> $recur_billed_amount,</nowiki>
'quantity' <nowiki>=> $override_quantity || $cust_pkg->quantity,</nowiki>
'details' <nowiki>=> \@details,</nowiki>
'discounts' <nowiki>=> [ @setup_discounts, @recur_discounts ],</nowiki>
'hidden' <nowiki>=> $part_pkg->hidden,</nowiki>
'freq' <nowiki>=> $part_pkg->freq,</nowiki>
};

Determine the start and end of the billing period. Usually it's “last bill to current bill”, or “current bill to next bill”, but there are a couple of special cases. Also record which part_pkg this line item was generated for, if it's an add-on package.

if ( $part_pkg->option('prorate_defer_bill',1)
and !$hash{last_bill} ) {
<nowiki># both preceding and upcoming, technically</nowiki>
$cust_bill_pkg->sdate( $cust_pkg->setup );
$cust_bill_pkg->edate( $cust_pkg->bill );
} elsif ( $part_pkg->recur_temporality eq 'preceding' ) {
$cust_bill_pkg->sdate( $hash{last_bill} );
$cust_bill_pkg->edate( $sdate - 86399 ); #60s*60m*24h-1
$cust_bill_pkg->edate( $time ) if $options{cancel};
} else { #if ( $part_pkg->recur_temporality eq 'upcoming' )
$cust_bill_pkg->sdate( $sdate );
$cust_bill_pkg->edate( $cust_pkg->bill );
<nowiki>#$cust_bill_pkg->edate( $time ) if $options{cancel};</nowiki>
}

$cust_bill_pkg->pkgpart_override($part_pkg->pkgpart)
unless $part_pkg->pkgpart == $real_pkgpart;

Record the results in the right places: add setup and recur to totals, give the cust_bill_pkg to the tax calculator, and add it to the line item array.

$$total_setup += $setup;
$$total_recur += $recur;

<nowiki>###</nowiki>
<nowiki># handle taxes</nowiki>
<nowiki>###</nowiki>

my $error = $tax_engine->add_sale($cust_bill_pkg);
return $error if $error;

$cust_bill_pkg->set_display(
part_pkg <nowiki>=> $part_pkg,</nowiki>
real_pkgpart => $real_pkgpart,
);

push @$cust_bill_pkgs, $cust_bill_pkg;

} #if $setup != 0 || $recur != 0

} #if $line_items

'';

}

And we're done!

Freeside:3:Documentation:Developer:Billing Internals

2017-02-08T05:41:04Z

Mark: Created page with "= FS::cust_main::Billing::bill = This is the main method for generating invoices. It's called on a single FS::cust_main, finds any packages that are due to be billed, and crea..."

= FS::cust_main::Billing::bill =
This is the main method for generating invoices. It's called on a single FS::cust_main, finds any packages that are due to be billed, and creates one or more invoices with the package charges.

It does not handle sending the invoice to the customer, charging their credit card, suspending packages if they're overdue, or anything else of that kind. Those are events run from collect(). See below.

== Initial setup ==
Grab the customer and options. Immediately exit if the customer is “complimentary”. Set the debug level, create a log context, and write a log entry.

my( $self, %options ) = @_;

return '' if $self->complimentary eq 'Y';

local($DEBUG) = $FS::cust_main::DEBUG if $FS::cust_main::DEBUG > $DEBUG;
my $log = FS::Log->new('FS::cust_main::Billing::bill');
my %logopt = (object => $self);

$log->debug('start', %logopt);
warn "$me bill customer ". $self->custnum. "\n"
if $DEBUG;

Set the billing clock. $time is the date we are “billing on”. For example, freeside-daily -y (bill packages a few days in the future so that invoices can be mailed out early) sets $time. This is almost, but not exactly, the same as changing the system clock.

$invoice_time is the date that will appear on invoices. Normally it's the same as $time; the -n option overrides that and forces it to be the real system time.

$cmp_time is the “comparison time”: a package gets billed if its next-bill date is <= $cmp_time. This is the same as $time unless '''next-bill-ignore-time''' is set, in which case we use the end of the day. Note that this does ''not'' affect certain other important time values such as prorate cutoffs.

Also parse the 'not_pkgpart' option to exclude specific packages from billing.

my $time = $options{'time'} || time;
my $invoice_time = $options{'invoice_time'} || $time;

my $cmp_time = ( $conf->exists('next-bill-ignore-time')
? day_end( $time )
: $time
);

$options{'not_pkgpart'} ||= {};
$options{'not_pkgpart'} = { map { $_ => 1 }
split(/\s*,\s*/, $options{'not_pkgpart'})
}
unless ref($options{'not_pkgpart'});

Block out interruptions, start a transaction, and lock the customer record (so that packages aren't ordered or status-changed, payments aren't recorded, etc. while doing this). If anything goes wrong during billing, we will rollback and return an error.

local $SIG{HUP} = 'IGNORE';
local $SIG{INT} = 'IGNORE';
local $SIG{QUIT} = 'IGNORE';
local $SIG{TERM} = 'IGNORE';
local $SIG{TSTP} = 'IGNORE';
local $SIG{PIPE} = 'IGNORE';

my $oldAutoCommit = $FS::UID::AutoCommit;
local $FS::UID::AutoCommit = 0;
my $dbh = dbh;

$log->debug('acquiring lock', %logopt);
warn "$me acquiring lock on customer ". $self->custnum. "\n"
if $DEBUG;

$self->select_for_update; #mutex

== Run pre-billing events. ==
Certain billing event actions are always run just before billing, because they affect the charges that will appear on the bill. These are:

* pkg_discount
* cust_pkg_fee, cust_fee, and pkg_fee
* The old “fee” event that created fees as one-time charges.

In this section we call do_cust_event() with a 'stage' parameter to specify only the pre-billing events.

$log->debug('running pre-bill events', %logopt);
warn "$me running pre-bill events for customer ". $self->custnum. "\n"
if $DEBUG;

my $error = $self->do_cust_event(
'debug' <nowiki>=> ( $options{'debug'} || 0 ),</nowiki>
'time' <nowiki>=> $invoice_time,</nowiki>
'check_freq' => $options{'check_freq'},
'stage' <nowiki>=> 'pre-bill',</nowiki>
)
unless $options{no_commit};
if ( $error ) {
$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return $error;
}

$log->debug('done running pre-bill events', %logopt);
warn "$me done running pre-bill events for customer ". $self->custnum. "\n"
if $DEBUG;

== Identify billable packages and add-on packages. ==
Set up space for the line items (%cust_bill_pkg), setup and recur totals, and a tax engine. Every time we add a line item to the invoice, we must push it to the line item array, add its setup and recur charges to the totals, and send it to the tax engine.

A single call to bill() can generate multiple invoices. Packages can be eligible or ineligible for automatic payment; that's implemented by splitting non-auto-payment packages off onto a separate invoice. So we'll (potentially) make two invoices here, and need a line items array, subtotals, etc. for each of them. If any package has the “separate_bill” flag, we'll make an additional invoice just for that package.

Also, if the set of packages to bill wasn't specified ('pkg_list'; it's usually not), then use all of the customer's non-canceled packages. We will check later for whether they're due for billing; that's complicated.

<nowiki>#keep auto-charge and non-auto-charge line items separate</nowiki>
my @passes = ( '', 'no_auto' );

my %cust_bill_pkg = map { $_ => [] } @passes;

<nowiki>###</nowiki>
<nowiki># find the packages which are due for billing, find out how much they are</nowiki>
<nowiki># & generate invoice database.</nowiki>
<nowiki>###</nowiki>

my %total_setup <nowiki>= map { my $z = 0; $_ => \$z; } @passes;</nowiki>
my %total_recur <nowiki>= map { my $z = 0; $_ => \$z; } @passes;</nowiki>

my @precommit_hooks = ();

$options{'pkg_list'} ||= [ $self->ncancelled_pkgs ]; <nowiki>#param checks?</nowiki>

my %tax_engines;
my $tax_is_batch = '';
foreach (@passes) {
$tax_engines{$_} = FS::TaxEngine->new(cust_main <nowiki>=> $self,</nowiki>
invoice_time => $invoice_time,
cancel <nowiki>=> $options{cancel},</nowiki>
estimate <nowiki>=> $options{estimate},</nowiki>
);
$tax_is_batch ||= $tax_engines{$_}->info->{batch};
}

Here we start the main loop over all packages. Skip any that are excluded by 'not_pkgpart', or by 'no_prepaid'. (freeside-daily sets this, as prepaid packages are billed by another mechanism.) And if there's an explicit 'pkg_list' then use only those.

foreach my $cust_pkg ( @{ $options{'pkg_list'} } ) {

next if $options{'not_pkgpart'}->{$cust_pkg->pkgpart};

my $part_pkg = $cust_pkg->part_pkg;

next if $options{'no_prepaid'} && $part_pkg->is_prepaid;

$log->debug('bill package '. $cust_pkg->pkgnum, %logopt);
warn " bill package ". $cust_pkg->pkgnum. "\n" if $DEBUG;

<nowiki>#? to avoid use of uninitialized value errors... ?</nowiki>
$cust_pkg->setfield('bill', '')
unless defined($cust_pkg->bill);

Call self_and_bill_linked() to find any billing add-on packages. Add-ons are additional package definitions (part_pkg records, plus all their accessories) that are linked to a “main” package definition, and generate additional charges on the bill. For example, a phone package that has a monthly fee plus long-distance usage billing would have an add-on voip_cdr package to do the usage billing.

Add-on packages always create separate cust_bill_pkg records, with the “pkgpart_override” field referencing the add-on package. Normally, these charges will show separately on the invoice, but they can be “bundled” with the main package by setting part_pkg_link.hidden. At this point we set a flag if this looks like a bundle package, as that will matter later.

my $real_pkgpart = $cust_pkg->pkgpart;
my %hash = $cust_pkg->hash;

<nowiki># we could implement this bit as FS::part_pkg::has_hidden, but we already</nowiki>
<nowiki># suffer from performance issues</nowiki>
$options{has_hidden} = 0;
my @part_pkg = $part_pkg->self_and_bill_linked;
$options{has_hidden} = 1 if ($part_pkg[1] && $part_pkg[1]->hidden);

== Create line items for each billable package. ==
As we create line items we'll push them onto $cust_bill_pkg{$pass} (either '' or 'no_auto'). First, balance transfers: if this package is being billed for the first time after a package change, and package balances are enabled, then credit the old package to zero out its balance, and make a line item to carry the balance forward to its successor package.

<nowiki># if this package was changed from another package,</nowiki>
<nowiki># and it hasn't been billed since then,</nowiki>
<nowiki># and package balances are enabled,</nowiki>
if ( $cust_pkg->change_pkgnum
and $cust_pkg->change_date >= ($cust_pkg->last_bill || 0)
and $cust_pkg->change_date < $invoice_time
and $conf->exists('pkg-balances') )
{
<nowiki># _transfer_balance will also create the appropriate credit</nowiki>
my @transfer_items = $self->_transfer_balance($cust_pkg);
<nowiki># $part_pkg[0] is the "real" part_pkg</nowiki>
my $pass = ($cust_pkg->no_auto || $part_pkg[0]->no_auto) ?
'no_auto' : '';
push @{ $cust_bill_pkg{$pass} }, @transfer_items;
<nowiki># treating this as recur, just because most charges are recur...</nowiki>
${$total_recur{$pass}} += $_->recur foreach @transfer_items;

<nowiki># currently not considering separate_bill here, as it's for </nowiki>
<nowiki># one-time charges only</nowiki>
}

Loop over the part_pkgs (first the “real” package definition, then the add-ons) and decide which invoice to put the charges on. Normally, $pass = '', which will put all charges on a single invoice, but no_auto packages have to go on another invoice, and each package with separate_bill needs yet another (so it will create entries in %total_setup, %total_recur, %cust_bill_pkg, etc.).

foreach my $part_pkg ( @part_pkg ) {

my $this_cust_pkg = $cust_pkg;
<nowiki># for add-on packages, copy the object to avoid leaking changes back to</nowiki>
<nowiki># the caller if pkg_list is in use; see RT#73607</nowiki>
if ( $part_pkg->get('pkgpart') != $real_pkgpart ) {
$this_cust_pkg = FS::cust_pkg->new({ %hash });
}

my $pass = '';
if ( $this_cust_pkg->separate_bill ) {
<nowiki># if no_auto is also set, that's fine. we just need to not have</nowiki>
<nowiki># invoices that are both auto and no_auto, and since the package</nowiki>
<nowiki># gets an invoice all to itself, it will only be one or the other.</nowiki>
$pass = $this_cust_pkg->pkgnum;
if (!exists $cust_bill_pkg{$pass}) { # it may not exist yet
push @passes, $pass;
$total_setup{$pass} = do { my $z = 0; \$z };
$total_recur{$pass} = do { my $z = 0; \$z };
<nowiki># it also needs its own tax context</nowiki>
$tax_engines{$pass} = FS::TaxEngine->new(
cust_main <nowiki>=> $self,</nowiki>
invoice_time => $invoice_time,
cancel <nowiki>=> $options{cancel},</nowiki>
estimate <nowiki>=> $options{estimate},</nowiki>
);
$cust_bill_pkg{$pass} = [];
}
} elsif ( ($this_cust_pkg->no_auto || $part_pkg->no_auto) ) {
$pass = 'no_auto';
}

The loop here checks whether the package is due for billing, and handles back-billing of multiple cycles. For example, if a monthly package's bill date is more than a month in the past, we'll bill as many months as necessary to bring the package up to date. The loop normally exits once the package bill date is in the future, but will exit after a single pass if we're billing on cancellation, or if freeside-daily -o is in use. It will also exit if there's an error or if the bill date isn't being incremented (as for a one-time charge).

_make_lines() is complex enough to have [#_make_lines its own section of the docs], but it generates a line item and does the following with it:

* appends it to the arrayref in 'line_items' (so, to $cust_bill_pkg{$pass}).
* adds its setup and recur fees to the running totals
* informs the tax engine about it (via the add_sale() method)
* increments the package's last-bill and next-bill dates
* returns an error if anything goes wrong

my $next_bill = $this_cust_pkg->getfield('bill') || 0;
my $error;
<nowiki># let this run once if this is the last bill upon cancellation</nowiki>
while ( $next_bill <= $cmp_time or $options{cancel} ) {
$error =
$self->_make_lines( 'part_pkg' <nowiki>=> $part_pkg,</nowiki>
'cust_pkg' <nowiki>=> $this_cust_pkg,</nowiki>
'precommit_hooks' <nowiki>=> \@precommit_hooks,</nowiki>
'line_items' <nowiki>=> $cust_bill_pkg{$pass},</nowiki>
'setup' <nowiki>=> $total_setup{$pass},</nowiki>
'recur' <nowiki>=> $total_recur{$pass},</nowiki>
'tax_engine' <nowiki>=> $tax_engines{$pass},</nowiki>
'time' <nowiki>=> $time,</nowiki>
'real_pkgpart' <nowiki>=> $real_pkgpart,</nowiki>
'options' <nowiki>=> \%options,</nowiki>
);

<nowiki># Stop if anything goes wrong</nowiki>
last if $error;

<nowiki># or if we're not incrementing the bill date.</nowiki>
last if ($this_cust_pkg->getfield('bill') || 0) == $next_bill;

<nowiki># or if we're letting it run only once</nowiki>
last if $options{cancel};

$next_bill = $this_cust_pkg->getfield('bill') || 0;

<nowiki>#stop if -o was passed to freeside-daily</nowiki>
last if $options{'one_recur'};
}
if ($error) {
$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return $error;
}

} #foreach my $part_pkg

} #foreach my $cust_pkg

== Identify applicable fees and tax adjustments and create line items. ==
At this point we have a list of package line items for each invoice the customer might receive. Now loop through the (possible) invoices and process non-package fees.

A “fee origin” (FS::FeeOrigin_Mixin) is an instruction or “order” to charge a fee<nowiki>; the classes are FS::cust_event_fee (fees generated by billing events, </nowiki>such as late fees) and FS::cust_pkg_reason_fee (unsuspension fees). When the fee is put on an invoice, the order is fulfilled; the billpkgnum of the fee line item is then recorded on the fee origin record.

The by_cust() method does a search for all fee origins for a customer; we limit it to records with billpkgnum = NULL to get fees that still need to be charged. Those are @pending_fees.

We process fees at this stage (after generating all package line items) because we need to know if the customer is going to receive an invoice or not. Fees can be defined (via the 'nextbill' flag) to be charged immediately, or on the customer's next bill. If all pending fees have 'nextbill' and there aren't any package charges on this invoice (@cust_bill_pkg is empty) then skip to the next invoice. If the customer has no package charges at all, then the 'nextbill' fees won't be charged at all on this billing date and will remain pending.

foreach my $pass (@passes) { # keys %cust_bill_pkg )

my @cust_bill_pkg = _omit_zero_value_bundles(@{ $cust_bill_pkg{$pass} });

warn "$me billing pass $pass\n"
<nowiki>#.Dumper(\@cust_bill_pkg)."\n"</nowiki>
if $DEBUG > 2;

<nowiki>###</nowiki>
<nowiki># process fees</nowiki>
<nowiki>###</nowiki>

my @pending_fees = FS::FeeOrigin_Mixin->by_cust($self->custnum,
hashref => { 'billpkgnum' => '' }
);
warn "$me found pending fees:\n".Dumper(\@pending_fees)."\n"
if @pending_fees and $DEBUG > 1;

<nowiki># determine whether to generate an invoice</nowiki>
my $generate_bill = scalar(@cust_bill_pkg) > 0;

foreach my $fee (@pending_fees) {
$generate_bill = 1 unless $fee->nextbill;
}

<nowiki># don't create an invoice with no line items, or where the only line </nowiki>
<nowiki># items are fees that are supposed to be held until the next invoice</nowiki>
next if !$generate_bill;

For each fee origin, do some things. Skip the fee if the fee definition is somehow inappropriate:

<nowiki># calculate fees...</nowiki>
my @fee_items;
foreach my $fee_origin (@pending_fees) {
my $part_fee = $fee_origin->part_fee;

<nowiki># check whether the fee is applicable before doing anything expensive:</nowiki>
<nowiki>#</nowiki>
<nowiki># if the fee def belongs to a different agent, don't charge the fee.</nowiki>
<nowiki># event conditions should prevent this, but just in case they don't,</nowiki>
<nowiki># skip the fee.</nowiki>
if ( $part_fee->agentnum and $part_fee->agentnum != $self->agentnum ) {
warn "tried to charge fee#".$part_fee->feepart .
" on customer#".$self->custnum." from a different agent.\n";
next;
}
<nowiki># also skip if it's disabled</nowiki>
next if $part_fee->disabled eq 'Y';

Then determine the “basis” for the fee, in case it's defined as a percentage of something. The fee origin may declare that the fee applies to a previous invoice (such as a finance charge). Otherwise, the fee applies to the ''current'' invoice. Since the current invoice doesn't exist yet, construct one temporarily that has the line items of the current invoice.

<nowiki># Decide which invoice to base the fee on.</nowiki>
my $cust_bill = $fee_origin->cust_bill;
if (!$cust_bill) {
<nowiki># Then link it to the current invoice. This isn't the real cust_bill</nowiki>
<nowiki># object that will be inserted--in particular there are no taxes yet.</nowiki>
<nowiki># If you want to charge a fee on the total invoice amount including</nowiki>
<nowiki># taxes, you have to put the fee on the next invoice.</nowiki>
$cust_bill = FS::cust_bill->new({
'custnum' <nowiki>=> $s</nowiki>elf->custnum,
'cust_bill_pkg' => \@cust_bill_pkg,
'charged' <nowiki>=> ${ $total_setup{$pass} } +</nowiki>
${ $total_recur{$pass} },
});
<nowiki># If the origin is for a specific package, then only apply the fee to</nowiki>
<nowiki># line items from that package.</nowiki>
if ( my $cust_pkg = $fee_origin->cust_pkg ) {
my @charge_fee_on_item;
my $charge_fee_on_amount = 0;
foreach (@cust_bill_pkg) {
if ($_->pkgnum == $cust_pkg->pkgnum) {
push @charge_fee_on_item, $_;
$charge_fee_on_amount += $_->setup + $_->recur;
}
}
$cust_bill->set('cust_bill_pkg', \@charge_fee_on_item);
$cust_bill->set('charged', $charge_fee_on_amount);
}

} # $cust_bill is now set

FS::part_fee->lineitem() asks the fee definition to make a cust_bill_pkg record for this fee, as applied to a specific invoice. After the invoice is inserted, we're going to need to record the billpkgnum on the fee origin record. FS::cust_bill_pkg->insert knows to do this.

Then append any fee line items to the invoice, add them to the running totals, and send them to the tax engine.

<nowiki># calculate the fee</nowiki>
my $fee_item = $part_fee->lineitem($cust_bill) or next;
<nowiki># link this so that we can clear the marker on inserting the line item</nowiki>
$fee_item->set('fee_origin', $fee_origin);
push @fee_items, $fee_item;

}

<nowiki># add fees to the invoice</nowiki>
foreach my $fee_item (@fee_items) {

push @cust_bill_pkg, $fee_item;
${ $total_setup{$pass} } += $fee_item->setup;
${ $total_recur{$pass} } += $fee_item->recur;

my $part_fee = $fee_item->part_fee;
my $fee_location = $self->ship_location; # I think?

my $error = $tax_engines{''}->add_sale($fee_item);

return $error if $error;

}

Add the special “postal invoice fee”. This was a per-invoice flat fee for sending customers a printed invoice; like all “old” fees it worked by adding a one-time charge. charge_postal_fee() is the method to do that. It adds a one-time package to the customer ''in the middle of billing'', after all their other packages have been billed, for exactly the same reason that other fees are processed at this point in billing. This feature is obsolete so I won't say any more.

<nowiki># XXX implementation of fees is supposed to make this go away...</nowiki>
if ( scalar( grep { $_->recur && $_->recur > 0 } @cust_bill_pkg) ||
!$conf->exists('postal_invoice-recurring_only')
)
{

my $postal_pkg = $self->charge_postal_fee();
if ( $postal_pkg && !ref( $postal_pkg ) ) {

$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return "can't charge postal invoice fee for customer ".
$self->custnum. ": $postal_pkg";

} elsif ( $postal_pkg ) {

my $real_pkgpart = $postal_pkg->pkgpart;
<nowiki># we could implement this bit as FS::part_pkg::has_hidden, but we already</nowiki>
<nowiki># suffer from performance issues</nowiki>
$options{has_hidden} = 0;
my @part_pkg = $postal_pkg->part_pkg->self_and_bill_linked;
$options{has_hidden} = 1 if ($part_pkg[1] && $part_pkg[1]->hidden);

foreach my $part_pkg ( @part_pkg ) {
my %postal_options = %options;
delete $postal_options{cancel};
my $error =
$self->_make_lines( 'part_pkg' <nowiki>=> $part_pkg,</nowiki>
'cust_pkg' <nowiki>=> $postal_pkg,</nowiki>
'precommit_hooks' <nowiki>=> \@precommit_hooks,</nowiki>
'line_items' <nowiki>=> \@cust_bill_pkg,</nowiki>
'setup' <nowiki>=> $total_setup{$pass},</nowiki>
'recur' <nowiki>=> $total_recur{$pass},</nowiki>
'tax_engine' <nowiki>=> $tax_engines{$pass},</nowiki>
'time' <nowiki>=> $time,</nowiki>
'real_pkgpart' <nowiki>=> $real_pkgpart,</nowiki>
'options' <nowiki>=> \%postal_options,</nowiki>
);
if ($error) {
$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return $error;
}
}

<nowiki># it's silly to have a zero value postal_pkg, but....</nowiki>
@cust_bill_pkg = _omit_zero_value_bundles(@cust_bill_pkg);

}

}

Tax adjustments (FS::cust_tax_adjustment) are manual adjustment records that work much like fee origins, but are created manually. If a tax adjustment exists for the customer, and doesn't have a billpkgnum yet, and an invoice is being generated, then create a line item with the amount and itemdesc specified in the tax adjustment. Note that this line item ''is a tax line item'': it has pkgnum = 0, feepart = NULL, and recur = 0. On that line item, set 'cust_tax_adjustment' to the tax adjustment record so that cust_bill_pkg->insert can record that the adjustment was billed.

<nowiki>#add tax adjustments</nowiki>
<nowiki>#XXX does this work with batch tax engines?</nowiki>
warn "adding tax adjustments...\n" if $DEBUG > 2;
foreach my $cust_tax_adjustment (
qsearch('cust_tax_adjustment', { 'custnum' <nowiki>=> $self->custnum,</nowiki>
'billpkgnum' => '',
}
)
) {

my $tax = sprintf('%.2f', $cust_tax_adjustment->amount );

my $itemdesc = $cust_tax_adjustment->taxname;
$itemdesc = '' if $itemdesc eq 'Tax';

push @cust_bill_pkg, new FS::cust_bill_pkg {
'pkgnum' <nowiki>=> 0,</nowiki>
'setup' <nowiki>=> $tax,</nowiki>
'recur' <nowiki>=> 0,</nowiki>
'sdate' <nowiki>=> '',</nowiki>
'edate' <nowiki>=> '',</nowiki>
'itemdesc' <nowiki>=> $itemdesc,</nowiki>
'itemcomment' => $cust_tax_adjustment->comment,
'cust_tax_adjustment' => $cust_tax_adjustment,
<nowiki>#'cust_bill_pkg_tax_location' => \@cust_bill_pkg_tax_location,</nowiki>
};

}

== Create the invoice record. ==
Before storing the record, calculate some totals:

* cust_bill.charged: the sum of charges on this invoice (taxes will be added later)
* billing_balance: the customer's balance before this invoice.
* previous_balance: the customer's balance immediately after their previous invoice, determined as (billing_balance + charged) on that invoice.

Note that ''billing_balance and previous_balance are obsolete''. They were historically used to create invoice summary sections (“Balance on previous invoice”, “Payments/credits since then”), but since 2014 we actually look at the transaction history. This method produces correct results if transactions get voided.

my $charged = sprintf('%.2f', ${ $total_setup{$pass} } + ${ $total_recur{$pass} } );

my $balance = $self->balance;

my $previous_bill = qsearchs({ 'table' <nowiki>=> 'cust_bill',</nowiki>
'hashref' <nowiki>=> { custnum=>$self->custnum },</nowiki>
'extra_sql' => 'ORDER BY _date DESC LIMIT 1',
});
my $previous_balance =
$previous_bill
? ( $previous_bill->billing_balance + $previous_bill->charged )
: 0;

Now store the record. FS::cust_bill::insert() will notice the arrayref of line items and insert all of them. Line items with 'fee_origin' or 'tax_adjustment' references will also trigger updates to those records.

The 'no_commit' option is for simulating billing, and will stop the record from being inserted. Currently this is used only with term prepayment discounts (which are no longer a supported feature) to figure out the “amount saved” over the prepaid term.

Note that (like almost anywhere else in Freeside) $FS::UID::AutoCommit = 0 will also prevent bill() from committing its changes. This is how quotations work: FS::quotation->estimate opens a transaction, inserts packages, runs bill(), and then reverts the whole transaction.

$log->debug('creating the new invoice', %logopt);
warn "creating the new invoice\n" if $DEBUG;
<nowiki>#create the new invoice</nowiki>
my $cust_bill = new FS::cust_bill ( {
'custnum' <nowiki>=> $self->custnum,</nowiki>
'_date' <nowiki>=> $invoice_time,</nowiki>
'charged' <nowiki>=> $charged,</nowiki>
'billing_balance' <nowiki>=> $balance,</nowiki>
'previous_balance' <nowiki>=> $previous_balance,</nowiki>
'invoice_terms' <nowiki>=> $options{'invoice_terms'},</nowiki>
'cust_bill_pkg' <nowiki>=> \@cust_bill_pkg,</nowiki>
'pending' <nowiki>=> 'Y', # clear this after doing taxes</nowiki>
} );

if (!$options{no_commit}) {
<nowiki># probably we ought to insert it as pending, and then rollback</nowiki>
<nowiki># without ever un-pending it</nowiki>
$error = $cust_bill->insert;
if ( $error ) {
$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return "can't create invoice for customer #". $self->custnum. ": $error";
}

}

== Calculate taxes and insert tax line items. ==
$tax_is_batch was previously set to true if we're using a batch tax processor. In that case, don't do any of this; just leave the invoice flagged as pending and FS::Cron::tax_batch will do the rest.

Otherwise: get the tax engine we created for this invoice pass, and call calculate_taxes(), passing the pending invoice. This will return an arrayref of line items, or throw an exception. If it throws an exception, rollback and exit.

<nowiki># calculate and append taxes</nowiki>
if ( ! $tax_is_batch) {
local $@;
my $arrayref = eval { $tax_engines{$pass}->calculate_taxes($cust_bill) };

if ( $@ ) {
$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return $@;
}

Append each tax line item to the invoice (directly, by setting invnum and inserting them) and add the charges to a running total. Tax line items have 'setup' but not 'recur' charges.

<nowiki># or should this be in TaxEngine?</nowiki>
my $total_tax = 0;
foreach my $taxline ( @$arrayref ) {
$total_tax += $taxline->setup;
$taxline->set('invnum' => $cust_bill->invnum); # just to be sure
push @cust_bill_pkg, $taxline; # for return_bill

if (!$options{no_commit}) {
my $error = $taxline->insert;
if ( $error ) {
$dbh->rollback if $oldAutoCommit;
return $error;
}
}

}

Add the total tax to the total invoice charges, remove the pending flag, and then update the invoice record. Then we're done.

<nowiki># add tax to the invoice amount and finalize it</nowiki>
${ $total_setup{$pass} } = sprintf('%.2f', ${ $total_setup{$pass} } + $total_tax);
$charged = sprintf('%.2f', $charged + $total_tax);
$cust_bill->set('charged', $charged);
$cust_bill->set('pending', '');

if (!$options{no_commit}) {
my $error = $cust_bill->replace;
if ( $error ) {
$dbh->rollback if $oldAutoCommit;
return $error;
}
}

} # if !$tax_is_batch
<nowiki># if it IS batch, then we'll do all this in process_tax_batch</nowiki>

After dealing with this invoice, push it to the 'return_bill' arrayref if there is one. This will allow the caller to see which invoices were created if they want.

push @{$options{return_bill}}, $cust_bill if $options{return_bill};

} #foreach my $pass ( keys %cust_bill_pkg )

Run any post-invoicing pre-commit hooks. These are currently used for only one thing: for packages with time-limited discounts, we need to increment the number of discount months used after completely billing the package, but before doing anything else. FS::part_pkg::discount_Mixin sets this up.

Then commit changes if needed, and return.

foreach my $hook ( @precommit_hooks ) {
eval {
&{$hook}; #($self) ?
} unless $options{no_commit};
if ( $@ ) {
$dbh->rollback if $oldAutoCommit && !$options{no_commit};
return "$@ running precommit hook $hook\n";
}
}

$dbh->commit or die $dbh->errstr if $oldAutoCommit && !$options{no_commit};

''; #no error
}

= FS::cust_main::Billing::_make_lines =
This is the other most important billing method. It takes a single package and package definition, and creates at most one line item for that package. This is called only from bill() and should probably never be used any other way, since it updates the package's billing dates.

== Gather parameters ==
$cust_pkg is the package. $part_pkg is the package definition, either the “real” one for this package or a billing add-on. We temporarily set the cust_pkg->pkgpart property to match it so that we can call cust_pkg methods that delegate to “$self->part_pkg”. We then copy all the cust_pkg's fields into %hash.

$time is the “billing as of” date. %options is all the options that were passed to bill().

The other $param{} elements are accumulators passed from outside so that this method can add stuff to them.

$cust_location here isn't used for anything, because of tax engine refactoring.

The reference to freq_override is one last attempt to make sure term discounts aren't set up in a radically absurd way.

$lineitems is for remembering if we've done anything that requires creating a line item; if not then we'll exit without pushing one to the array.

@details will be used for any detail records that we attach to this line item. Those are additional lines of text shown below the line item on the invoice.

$cmp_time, as in bill(), is either “right now” or “the end of today” depending on the '''next-bill-ignore-time''' option.

sub _make_lines {
my ($self, %params) = @_;

local($DEBUG) = $FS::cust_main::DEBUG if $FS::cust_main::DEBUG > $DEBUG;

my $part_pkg = $params{part_pkg} or die "no part_pkg specified";
my $cust_pkg = $params{cust_pkg} or die "no cust_pkg specified";
my $cust_location = $cust_pkg->tax_location;
my $precommit_hooks = $params{precommit_hooks} or die "no precommit_hooks specified";
my $cust_bill_pkgs = $params{line_items} or die "no line buffer specified";
my $total_setup = $params{setup} or die "no setup accumulator specified";
my $total_recur = $params{recur} or die "no recur accumulator specified";
my $time = $params{'time'} or die "no time specified";
my (%options) = %{$params{options}};

my $tax_engine = $params{tax_engine};

if ( $part_pkg->freq ne '1' and ($options{'freq_override'} || 0) > 0 ) {
<nowiki># this should never happen</nowiki>
die 'freq_override billing attempted on non-monthly package '.
$cust_pkg->pkgnum;
}

my $dbh = dbh;
my $real_pkgpart = $params{real_pkgpart};
my %hash = $cust_pkg->hash;
my $old_cust_pkg = new FS::cust_pkg \%hash;

my @details = ();
my $lineitems = 0;

$cust_pkg->pkgpart($part_pkg->pkgpart);

my $cmp_time = ( $conf->exists('next-bill-ignore-time')
? day_end( $time )
: $time
);

== Charge setup fee and set setup date ==
Setup fees are charged the first time a package is billed. %setup_param will be passed to the calc_setup() method; because of how discounts work, it needs to know if this is an add-on package, and to report if it's applying any discounts. $setup and $unitsetup will become those fields on the cust_bill_pkg record.

The conditions for charging the setup fee are exactly as described in the inline comment. Note that a package that's canceled before first billing will never be charged a setup fee (or any other fee). We assume the package was ordered by mistake.

Note also that if the package has an expire date <= $cmp_time then it won't be charged. Normally the package would be canceled already. This check is for the case where the package isn't expired yet (in real time) but is being billed for a date after it will expire. For example, if a package has a next bill date of Feb 1, but an expiration date of Jan 31, and you're running freeside-daily -y 3 to print invoices 3 days in advance, then billing will trigger for the customer on Jan 28. The package won't be canceled yet. However, we should ''not'' preprint an invoice for this package because it ''will'' expire before its real billing date arrives. This is a real case that has come up.

Possibly we should check adjourn dates also, since a suspended package is not supposed to get set up.

<nowiki>###</nowiki>
<nowiki># bill setup</nowiki>
<nowiki>###</nowiki>

my $setup = 0;
my $unitsetup = 0;
my @setup_discounts = ();
my %setup_param = ( 'discounts' <nowiki>=> \@setup_discounts,</nowiki>
'real_pkgpart' <nowiki>=> $params{real_pkgpart}</nowiki>
);
my $setup_billed_currency = '';
my $setup_billed_amount = 0;
<nowiki># Conditions for setting setup date and charging the setup fee:</nowiki>
<nowiki># - this is not a recurring-only billing run</nowiki>
<nowiki># - and the package is not currently being canceled</nowiki>
<nowiki># - and, unless we're specifically told otherwise via 'resetup':</nowiki>
<nowiki># </nowiki> - it doesn't already HAVE a setup date
<nowiki># </nowiki> - or a start date in the future
<nowiki># </nowiki> - and it's not suspended
<nowiki># - and it doesn't have an expire date in the past</nowiki>
<nowiki>#</nowiki>
<nowiki># The "disable_setup_suspended" option is now obsolete; we never set the</nowiki>
<nowiki># setup date on a suspended package.</nowiki>
if ( ! $options{recurring_only}
and ! $options{cancel}
and ( $options{'resetup'}
|| ( ! $cust_pkg->setup
&& ( ! $cust_pkg->start_date
|| $cust_pkg->start_date <= $cmp_time
)
&& ( ! $cust_pkg->getfield('susp') )
)
)
and ( ! $cust_pkg->expire
|| $cust_pkg->expire > $cmp_time )
)
{

warn " bill setup\n" if $DEBUG > 1;

If 'waive_setup' is set on the package then skip this step.

Calculate the setup fee. FS::cust_pkg->calc_setup just delegates to its part_pkg (which we've overridden if we're processing an add-on package right now). The following part_pkg classes have a calc_setup method:

* flat and recur_Common: Calls prorate_setup(), about which [#Deferred prorate see below]. Adds any “additional_info” strings to @details (this is used for putting notes on one-time charges). Calls base_setup() to get the setup fee from the package definition, then calc_discount() to apply any discounts, then multiplies the result by quantity. Almost all packages use this logic.
* currency_fixed: Calls prorate_setup(), then base_setup() (which does the currency conversion). Oddly, it doesn't multiply by quantity; this is probably a bug, since it does multiply by quantity in calc_recur().
* delayed_Mixin: If the 'delay_setup' option is off, pushes back the package bill date by some number of days so that the start of recurring billing is delayed. Then calls whatever calc_setup() method would otherwise apply.
* rt_field and (the obsolete) voip_sqlradacct: Returns the 'setup_fee' package option, without multiplying by quantity or applying discounts.

Also calculate the per-unit setup fee (for displaying on the invoice). The base_setup() method, by definition, returns the non-discounted per-unit setup fee.

Currency packages also return (via $setup_param) the currency and amount before doing conversion; we remember those here and will put them on the invoice later.

unless ( $cust_pkg->waive_setup ) {
$lineitems++;

$setup = eval { $cust_pkg->calc_setup( $time, \@details, \%setup_param ) };
return "$@ running calc_setup for $cust_pkg\n"
if $@;

<nowiki># Only increment unitsetup here if there IS a setup fee.</nowiki>
<nowiki># prorate_defer_bill may cause calc_setup on a setup-stage package</nowiki>
<nowiki># to return zero, and the setup fee to be charged later. (This happens</nowiki>
<nowiki># when it's first billed on the prorate cutoff day. RT#31276.)</nowiki>
if ( $setup ) {
$unitsetup = $cust_pkg->base_setup()
|| $setup; #XXX uuh
}

if ( $setup_param{'billed_currency'} ) {
$setup_billed_currency = delete $setup_param{'billed_currency'};
$setup_billed_amount <nowiki>= delete $setup_param{'billed_amount'};</nowiki>
}
}

Assign the setup date. The package may already have a setup date (because of 'resetup'). Otherwise, if it has a designated start date, use that (and clear the start date). Otherwise, it started billing now so set it to now.

if ( $cust_pkg->get('setup') ) {
<nowiki># don't change it</nowiki>
} elsif ( $cust_pkg->get('start_date') ) {
<nowiki># this allows start_date to be used to set the first bill date</nowiki>
$cust_pkg->set('setup', $cust_pkg->get('start_date'));
} else {
<nowiki># if unspecified, start it right now</nowiki>
$cust_pkg->set('setup', $time);
}

$cust_pkg->setfield('start_date', '')
if $cust_pkg->start_date;

}

=== Deferred prorate ===
This is an optional mode for prorate packages (and flat monthly packages using sync_bill_date), enabled by the prorate_defer_bill option. Rather than charging a fractional month at setup time, the package is billed on the first cutoff day for a full month ''plus'' the fractional month preceding the cutoff day.

In packages that are eligible for deferred prorate, calc_setup() calls the calc_prorate() method, which checks whether the flag is present. If so, calc_prorate sets the setup date (to now) and the next bill date (to the upcoming cutoff day) but doesn't set last_bill because the package hasn't really been billed. Then it returns a true value, which tells calc_setup ''not'' to charge a setup fee. The package will then not be charged a recurring fee either, because the next bill date is in the future.

(If the setup date is itself a cutoff day, then calc_prorate will ignore the flag and bill the package for a full month as usual. It is highly recommended that you use the prorate_round_day option in combination with this so that the “full month” really is a full month.)

When the first recurring billing date arrives, calc_recur() notices that there's a bill date but no last_bill, and charges for the partial month in the past and full month in the future. (The start and end dates on the line item will reflect this.) It will also call calc_setup() and apply any setup fee defined by the package.

== Charge the recurring fee and adjust the bill date ==
The comment here explains the logic for whether to charge a recurring fee.

At this point, if it still has a start_date, then either the start_date is in the future or the package couldn't start billing for some other reason (like that it's on hold).

See the note above about the expire date; this part is slightly different in that if the package actually ''has'' expired and we're billing it on cancellation, then charging a recurring fee is allowed.

<nowiki>###</nowiki>
<nowiki># bill recurring fee</nowiki>
<nowiki>### </nowiki>

my $recur = 0;
my $unitrecur = 0;
my @recur_discounts = ();
my $recur_billed_currency = '';
my $recur_billed_amount = 0;
my $sdate;

my $override_quantity;

<nowiki># Conditions for billing the recurring fee:</nowiki>
<nowiki># - the package doesn't have a future start date</nowiki>
<nowiki># - and it's not suspended</nowiki>
<nowiki># </nowiki> - unless suspend_bill is enabled on the package or package def
<nowiki># </nowiki> - but still not, if the package is on hold
<nowiki># </nowiki> - or it's suspended for a delayed cancellation
<nowiki># - and its next bill date is in the past</nowiki>
<nowiki># </nowiki> - or it doesn't have a next bill date yet
<nowiki># </nowiki> - or it's a one-time charge
<nowiki># </nowiki> - or it's a CDR plan with the "bill_every_call" option
<nowiki># </nowiki> - or it's being canceled
<nowiki># - and it doesn't have an expire date in the past (this can happen with</nowiki>
<nowiki># </nowiki> advance billing)
<nowiki># </nowiki> - again, unless it's being canceled
if ( ! $cust_pkg->start_date
and
( ! $cust_pkg->susp
|| ( $cust_pkg->susp != $cust_pkg->order_date
&& ( $cust_pkg->option('suspend_bill',1)
|| ( $part_pkg->option('suspend_bill', 1)
&& ! $cust_pkg->option('no_suspend_bill',1)
)
)
)
|| $cust_pkg->is_status_delay_cancel
)
and
( $part_pkg->freq ne '0' && ( $cust_pkg->bill || 0 ) <= $cmp_time )
|| ( $part_pkg->plan eq 'voip_cdr'
&& $part_pkg->option('bill_every_call')
)
|| $options{cancel}

and
( ! $cust_pkg->expire
|| $cust_pkg->expire > $cmp_time
|| $options{cancel}
)
) {

Reset the package's usage cap if it has one:

* For voip_cdr, this is for “usage pools”. It clears all cdr_cust_pkg_usage records from the package and resets the cust_pkg_usage's minutes remaining.
* For flat (and other flat-like packages), this finds svc_accts attached to the package and resets their byte usage counters. This is for RADIUS services with data caps.

<nowiki># XXX should this be a package event? </nowiki> probably. events are called
<nowiki># at collection time at the moment, though...</nowiki>
$part_pkg->reset_usage($cust_pkg, 'debug'=>$DEBUG)
if $part_pkg->can('reset_usage') && !$options{'no_usage_reset'};
<nowiki>#don't want to reset usage just cause we want a line item??</nowiki>
<nowiki>#&& $part_pkg->pkgpart == $real_pkgpart;</nowiki>

At this point we've decided to charge a recurring fee, so increment $lineitems.

“$sdate” is the effective billing date. It represents the end of the previous cycle and the start of the next. The next bill date will be set to $sdate + one billing period.

For a new package, $sdate is the setup date (so, the start date if there was one); if it's been billed before then it's the billing date.

Prorate math will normally charge for the period from $sdate to the next prorate cycle date after that. After that, it will set the next bill date to the prorate cycle date, so that future billing cyles run from one prorate cycle date to the next. The result is that the customer is charged for a partial month, then for full months after that.

$increment_next_bill is, yes, a flag saying to increment the bill date. We won't increment for one-time charges, or packages that are being canceled. The check for the bill date <= $cmp_time is now redundant, since we won't bill the package at all.

warn " bill recur\n" if $DEBUG > 1;
$lineitems++;

<nowiki># XXX shared with $recur_prog</nowiki>
$sdate = ( $options{cancel} ? $cust_pkg->last_bill : $cust_pkg->bill )
|| $cust_pkg->setup
|| $time;

<nowiki>#over two params! </nowiki> lets at least switch to a hashref for the rest...
my $increment_next_bill = ( $part_pkg->freq ne '0'
&& ( $cust_pkg->getfield('bill') || 0 ) <= $cmp_time
&& !$options{cancel}
);

The most important part of the entire module: calculate the recurring fee. Call FS::part_pkg::calc_recur (or calc_cancel, if we're canceling the package).

This is ''not a well-behaved interface''. The arguments are all passed by reference (they're used to pass values back out), and they include the contents of %setup_param, so packages can use the %param hashref argument to pass notes between the calc_setup and calc_recur stages. Discounts, especially, use this.

* $sdate is the effective bill date, @details is the array of line item details, and %param contains everything else.
* 'discounts' is an arrayref of objects that will be inserted with the line item to track which discounts applied.
* 'precommit_hooks' and 'real_pkgpart' are used to keep track of the months remaining on term-limited discounts: if real_pkgpart matches the pkgpart of the package, then it's not an add-on package, so a month (or other amount) should be deducted. In that case a callback will be appended to precommit_hooks to do that at the end.
* 'freq_override' should be ignored.
* 'setup_fee' allows calc_recur to charge a setup fee. This seems bizarre but is needed for certain prorate cases.
* 'increment_next_bill' is checked in recur_Common to decide whether to apply the fixed recurring charge for voip_cdr and other usage-based packages. The intent seems to be to not bill those charges on cancellation.

my %param = ( %setup_param,
'precommit_hooks' <nowiki>=> $precommit_hooks,</nowiki>
'increment_next_bill' => $increment_next_bill,
'discounts' <nowiki>=> \@recur_discounts,</nowiki>
'real_pkgpart' <nowiki>=> $real_pkgpart,</nowiki>
'freq_override' => $options{freq_override} || '',
'setup_fee' <nowiki>=> 0,</nowiki>
);

my $method = $options{cancel} ? 'calc_cancel' : 'calc_recur';

<nowiki># There may be some part_pkg for which this is wrong. </nowiki> Only those
<nowiki># which can_discount are supported.</nowiki>
<nowiki># (the UI should prevent adding discounts to these at the moment)</nowiki>

warn "calling $method on cust_pkg ". $cust_pkg->pkgnum.
" for pkgpart ". $cust_pkg->pkgpart.
" with params ". join(' / ', map "$_=>$param{$_}", keys %param). "\n"
if $DEBUG > 2;

$recur = eval { $cust_pkg->$method( \$sdate, \@details, \%param ) };
return "$@ running $method for $cust_pkg\n"
if ( $@ );

Handle a special case where calc_cancel() finds that the package isn't eligible to be billed on cancellation. This affects the decision about whether to create a line item at all.

if ($recur eq 'NOTHING') {
<nowiki># then calc_cancel (or calc_recur but that's not used) has declined to</nowiki>
<nowiki># generate a recurring lineitem at all. treat this as zero, but also </nowiki>
<nowiki># try not to generate a lineitem.</nowiki>
$recur = 0;
$lineitems--;
}

Process more things returned by calc_recur. Ask the package for the unit recurring charge (passing $sdate because packages with an intro rate will report a different amount in the intro period).

If calc_recur did a currency conversion, record how it happened.

If calc_recur reported a quantity, record that quantity and adjust unitrecur. sql_external packages can do this.

<nowiki>#base_cancel???</nowiki>
$unitrecur = $cust_pkg->base_recur( \$sdate ) || $recur; #XXX uuh, better

if ( $param{'billed_currency'} ) {
$recur_billed_currency = delete $param{'billed_currency'};
$recur_billed_amount <nowiki>= delete $param{'billed_amount'};</nowiki>
}

if ( $param{'override_quantity'} ) {
$override_quantity = $param{'override_quantity'};
$unitrecur = $recur / $override_quantity;
}

The other most important part of the module: increment the next bill date so that the package cycles.

As always in Freeside, adding some period to a bill date is done by the FS::part_pkg->add_freq method.

Deal with the case where this is a supplemental package. A supplemental package is “geared” to its main package: there's a fixed frequency ratio between them so that every N cycles they bill on the same day. If it's an integer multiple then that's easy (just add the main package period that man times). Otherwise, check whether the next bill is when they're due to sync up, and if so, sync them. This uses a rather crude “they're within 15 days” heuristic.

Otherwise, just call add_freq() using this package's frequency. Note that if the package is being prorated, $sdate will already have been moved to a prorate cycle date so that adding one month yields the right result.

if ( $increment_next_bill ) {

my $next_bill;

if ( my $main_pkg = $cust_pkg->main_pkg ) {
<nowiki># supplemental package</nowiki>
<nowiki># to keep in sync with the main package, simulate billing at </nowiki>
<nowiki># its frequency</nowiki>
my $main_pkg_freq = $main_pkg->part_pkg->freq;
my $supp_pkg_freq = $part_pkg->freq;
if ( $supp_pkg_freq == 0 or $main_pkg_freq == 0 ) {
<nowiki># the UI should prevent setting up packages like this, but just</nowiki>
<nowiki># in case</nowiki>
return "unable to calculate supplemental package period ratio";
}
my $ratio = $supp_pkg_freq / $main_pkg_freq;
if ( $ratio == int($ratio) ) {
<nowiki># simple case: main package is X months, supp package is X*A months,</nowiki>
<nowiki># advance supp package to where the main package will be in A cycles.</nowiki>
$next_bill = $sdate;
for (1..$ratio) {
$next_bill = $part_pkg->add_freq( $next_bill, $main_pkg_freq );
}
} else {
<nowiki># harder case: main package is X months, supp package is Y months.</nowiki>
<nowiki># advance supp package by Y months. then if they're within half a </nowiki>
<nowiki># month of each other, resync them. this may result in the period</nowiki>
<nowiki># not being exactly Y months.</nowiki>
$next_bill = $part_pkg->add_freq( $sdate, $supp_pkg_freq );
my $main_next_bill = $main_pkg->bill;
if ( $main_pkg->bill <= $time ) {
<nowiki># then the main package has not yet been billed on this cycle;</nowiki>
<nowiki># predict what its bill date will be.</nowiki>
$main_next_bill =
$part_pkg->add_freq( $main_next_bill, $main_pkg_freq );
}
if ( abs($main_next_bill - $next_bill) < 86400*15 ) {
$next_bill = $main_next_bill;
}
}

} else {
<nowiki># the normal case, not a supplemental package</nowiki>
$next_bill = $part_pkg->add_freq($sdate, $options{freq_override} || 0);
return "unparsable frequency: ".
($options{freq_override} || $part_pkg->freq)
if $next_bill == -1;
}

Set the “last bill” date to either the previous value of “next bill” or the setup date.

Also, if there's any second-pass setup fee, apply that to the line item.

'discount_left_setup' is a register used by the discount code for any flat-amount discount that's available for applying to the setup fee. Apply it here. This probably should set up a cust_bill_pkg_discount record but it's such an obscure case that we haven't needed it yet.

<nowiki>#pro-rating magic - if $recur_prog fiddled $sdate, want to use that</nowiki>
<nowiki># only for figuring next bill date, nothing else, so, reset $sdate again</nowiki>
<nowiki># here</nowiki>
$sdate = $cust_pkg->bill || $cust_pkg->setup || $time;
<nowiki>#no need, its in $hash{last_bill}# my $last_bill = $cust_pkg->last_bill;</nowiki>
$cust_pkg->last_bill($sdate);

$cust_pkg->setfield('bill', $next_bill );

}

if ( $param{'setup_fee'} ) {
<nowiki># Add an additional setup fee at the billing stage.</nowiki>
<nowiki># Used for prorate_defer_bill.</nowiki>
$setup += $param{'setup_fee'};
$unitsetup = $cust_pkg->base_setup();
$lineitems++;
}

if ( defined $param{'discount_left_setup'} ) {
foreach my $discount_setup ( values %{$param{'discount_left_setup'}} ) {
$setup -= $discount_setup;
}
}
} # end of recurring fee

warn "\$setup is undefined" unless defined($setup);
warn "\$recur is undefined" unless defined($recur);
warn "\$cust_pkg->bill is undefined" unless defined($cust_pkg->bill);

== Construct a cust_bill_pkg record ==
If $lineitems is zero then billing was entirely skipped and there's nothing to do.

Otherwise, save the cust_pkg record with the new bill and last_bill dates. Only do that when processing the base package; add-on packages don't increment the bill date.

Clean up $setup and $recur. The 'allow_negative_charges' config is no longer used.

<nowiki>###</nowiki>
<nowiki># If there's line items, create em cust_bill_pkg records</nowiki>
<nowiki># If $cust_pkg has been modified, update it (if we're a real pkgpart)</nowiki>
<nowiki>###</nowiki>

if ( $lineitems ) {

if ( $cust_pkg->modified && $cust_pkg->pkgpart == $real_pkgpart ) {
<nowiki># hmm.. and if just the options are modified in some weird price plan?</nowiki>

warn " package ". $cust_pkg->pkgnum. " modified; updating\n"
if $DEBUG >1;

my $error = $cust_pkg->replace( $old_cust_pkg,
'depend_jobnum'=>$options{depend_jobnum},
'options' => { $cust_pkg->options },
)
unless $options{no_commit};
return "Error modifying pkgnum ". $cust_pkg->pkgnum. ": $error"
if $error; #just in case
}

$setup = sprintf( "%.2f", $setup );
$recur = sprintf( "%.2f", $recur );
if ( $setup < 0 && ! $conf->exists('allow_negative_charges') ) {
return "negative setup $setup for pkgnum ". $cust_pkg->pkgnum;
}
if ( $recur < 0 && ! $conf->exists('allow_negative_charges') ) {
return "negative recur $recur for pkgnum ". $cust_pkg->pkgnum;
}

One more check. If the setup and recurring fees are both zero, we might skip creating a line item. It depends. Create a zero-dollar line item if:

* The setup or recur is zero ''because it was discounted to zero'', and the '''discount-show-always''' config is on. The invoice will then show the applied discount.
* This is the main package in a ''bundle''. Its add-on packages will add hidden line items for the charges, but we need this one so that the bundle is shown with the correct package name.
* Either the cust_pkg or the part_pkg has the 'setup_show_zero' or 'recur_show_zero' flag. (It's not clear to me why these are separate flags, since they have exactly the same effect.)

my $discount_show_always = $conf->exists('discount-show-always')
&& ( ($setup == 0 && scalar(@setup_discounts))
|| ($recur == 0 && scalar(@recur_discounts))
);

if ( $setup != 0
|| $recur != 0
|| (!$part_pkg->hidden && $options{has_hidden}) #include some $0 lines
|| $discount_show_always
|| ($setup == 0 && $cust_pkg->_X_show_zero('setup'))
|| ($recur == 0 && $cust_pkg->_X_show_zero('recur'))
)
{

warn " charges (setup=$setup, recur=$recur); adding line items\n"
if $DEBUG > 1;

Find any package details that are set to display as “invoice details” and add them to the list.

Then actually create the cust_bill_pkg. Huzzah!

my @cust_pkg_detail = map { $_->detail } $cust_pkg->cust_pkg_detail('I');
if ( $DEBUG > 1 ) {
warn " adding customer package invoice detail: $_\n"
foreach @cust_pkg_detail;
}
push @details, @cust_pkg_detail;
my $cust_bill_pkg = new FS::cust_bill_pkg {
'pkgnum' <nowiki>=> $cust_pkg->pkgnum,</nowiki>
'setup' <nowiki>=> $setup,</nowiki>
'unitsetup' <nowiki>=> sprintf('%.2f', $unitsetup),</nowiki>
'setup_billed_currency' => $setup_billed_currency,
'setup_billed_amount' <nowiki>=> $setup_billed_amount,</nowiki>
'recur' <nowiki>=> $recur,</nowiki>
'unitrecur' <nowiki>=> sprintf('%.2f', $unitrecur),</nowiki>
'recur_billed_currency' => $recur_billed_currency,
'recur_billed_amount' <nowiki>=> $recur_billed_amount,</nowiki>
'quantity' <nowiki>=> $override_quantity || $cust_pkg->quantity,</nowiki>
'details' <nowiki>=> \@details,</nowiki>
'discounts' <nowiki>=> [ @setup_discounts, @recur_discounts ],</nowiki>
'hidden' <nowiki>=> $part_pkg->hidden,</nowiki>
'freq' <nowiki>=> $part_pkg->freq,</nowiki>
};

Determine the start and end of the billing period. Usually it's “last bill to current bill”, or “current bill to next bill”, but there are a couple of special cases. Also record which part_pkg this line item was generated for, if it's an add-on package.

if ( $part_pkg->option('prorate_defer_bill',1)
and !$hash{last_bill} ) {
<nowiki># both preceding and upcoming, technically</nowiki>
$cust_bill_pkg->sdate( $cust_pkg->setup );
$cust_bill_pkg->edate( $cust_pkg->bill );
} elsif ( $part_pkg->recur_temporality eq 'preceding' ) {
$cust_bill_pkg->sdate( $hash{last_bill} );
$cust_bill_pkg->edate( $sdate - 86399 ); #60s*60m*24h-1
$cust_bill_pkg->edate( $time ) if $options{cancel};
} else { #if ( $part_pkg->recur_temporality eq 'upcoming' )
$cust_bill_pkg->sdate( $sdate );
$cust_bill_pkg->edate( $cust_pkg->bill );
<nowiki>#$cust_bill_pkg->edate( $time ) if $options{cancel};</nowiki>
}

$cust_bill_pkg->pkgpart_override($part_pkg->pkgpart)
unless $part_pkg->pkgpart == $real_pkgpart;

Record the results in the right places: add setup and recur to totals, give the cust_bill_pkg to the tax calculator, and add it to the line item array.

$$total_setup += $setup;
$$total_recur += $recur;

<nowiki>###</nowiki>
<nowiki># handle taxes</nowiki>
<nowiki>###</nowiki>

my $error = $tax_engine->add_sale($cust_bill_pkg);
return $error if $error;

$cust_bill_pkg->set_display(
part_pkg <nowiki>=> $part_pkg,</nowiki>
real_pkgpart => $real_pkgpart,
);

push @$cust_bill_pkgs, $cust_bill_pkg;

} #if $setup != 0 || $recur != 0

} #if $line_items

'';

}

And we're done!

Category:Freeside 5

2017-01-23T17:54:47Z

Mark: Created page with "Pages related to Freeside version 5 (the master branch as of 2017)."

Pages related to Freeside version 5 (the master branch as of 2017).

Freeside:5:Documentation:Tower mapping

2017-01-23T17:54:09Z

Mark:

This document describes the tower coverage features on the 5.x branch, which has not yet been released. The tower coverage system works, but further development is planned. Red text is for notes on future improvements.

= Tower coverage calculation =
The goals of tower coverage calculation are:

* ''Mapping'': estimating a tower's coverage area, for marketing, deployment planning, and regulatory reporting.
* ''Prequalification'': estimating the quality of signal available at a customer site, prior to doing an on-site survey.

These require the same calculation: evaluating a signal strength function at a geographic point, using the tower's transmitter characteristics and a terrain map, and comparing to a minimum specified signal strength:

<math>{P}_{\mathit{RX},\mathit{\min }}\le {P}_{\mathit{TX}}+{G}_{\mathit{TX}}-{L}_{\mathit{path}}+{G}_{\mathit{RX}}</math>

where ''PTX'' is the transmitter power, ''GTX'' and ''GRX'' are the transmitter and receiver antenna gains, and ''Lpath'' is the path loss. For an unobstructed path length ''d ''and a frequency ''f'' , the path loss is:

<math>{L}_{\mathit{path}}=20{\log }_{10}4\mathrm{\pi }df/c\approx 36.6\mathrm{dB}+20{\log }_{10}f-20{\log }_{10}d</math>

where ''f'' is in MHz and ''d'' is in miles. For a partially obstructed path, this is more complicated; note that the path is significantly wider than the physical antenna due to diffraction. Consult your radio engineer for details.

Tower configuration is in '''Configuration / Services / Wireless broadband / Towers'''.

= Tower/sector schema =
A record in “tower” represents a site where broadband equipment is installed, including:

* Latitude and longitude
* Tower elevation

A record in “tower_sector” represents a transmitter and antenna installed at a tower. Normally, each wireless broadband service is linked to a sector. For each sector, Freeside tracks:

* IP address
* Height above ground level in feet
* Antenna direction (azimuth and “downtilt”, i.e. elevation)
* Beam width (horizontal and vertical)
* Minimum signal margin
* Power output, in dBm
* Line loss (in cables and connectors), in dB
* Antenna gain, in dB
* “Low-quality” and “high-quality” signal margins. Note that this represents ''maximum path loss'' in dB, not signal strength as such. For example, if db_low = 100, then for “low quality” service to be available, ''PTX'' + ''GTX'' + ''GRX'' − ''PRX,min'' must be at least 100 dB. It is normal for db_low to be larger than db_high. Possibly these should both be export parameters rather than being set per sector. Allowing more than two levels might also be useful.
* “image”, a PNG graphic of the coverage map
* The geographic bounds of the image: west, south, east, and north.
* hardware_typenum, a foreign key to the hardware_type table.
* title, a string field for storing an external ID.

Coverage maps are stored in “sector_coverage”. A separate record is created for each minimum signal level; the intent is that they can be used as map layers. Each one contains:

* sectornum (tower_sector foreign key)
* db_loss, the level of signal loss represented by this map layer
* geometry, a GeoJSON polygon for the region where the signal loss is at most db_loss.

If you move the database into PostGIS, with tower_sector.image becoming a raster and sector_coverage.geometry becoming a polygon layer, then a simple prequalification looks like:

dbh->selectall_arrayref('SELECT sectornum, db_loss FROM sector_coverage WHERE ST_Contains(sector_coverage.geometry, ST_Point(?, ?)) AND db_loss > ? ORDER BY db_loss DESC', {}, $longitude, $latitude, $max_loss)

= Tower mapping with Splat: process_generate_coverage =
Inserting a new tower_sector will queue a process_generate_coverage job for the sector. Editing the height, antenna position or beamwidth, or signal loss levels will also queue this job (from the replace() method). Changing a tower's coordinates will queue process_generate_coverage() for all sectors on the tower.

CloudRF replaces most of this; supposedly they will provide an ESRI shapefile, which we can pull into OGR and convert into a stack of sector_coverage records.

process_generate_coverage makes a temporary directory and creates a Map::Splat object, passing the tower coordinates, antenna height, azimuth and downtilt, vertical and horizontal beamwidth, and the two signal levels to calculate (db_low and db_high). It calls the calculate() method to start generating the map.

Splat is a standalone program that operates on a set of text files. Map::Splat, our wrapper library for it, writes the input files into the temp directory:

* The tower location file (QTH). This only contains the tower coordinates and antenna height.
* The model parameter file (LRP). This contains the frequency, and several static parameters (in particular, the model's safety margin may need to be adjustable).
* The loss contour file (LCF). This maps signal loss levels (dB) to 24-bit colors. Splat always uses white for areas with “no signal” (that is, above the highest specified loss level) and won't accept a color list containing black, or with more than 32 colors. Map::Splat accepts a list of desired signal levels, and spreads them out in the red channel from #04 to #FC. Here, we only have two signal levels, so they'll be at the endpoints.
* The antenna profiles in the azimuth plane (AZ) and the elevation plane (EL). These start with the antenna orientation (which is applied as a rotation to the entire data set) and then list several angular positions and field strengths (relative to the on-axis field strength). Currently we use a simple approximation where the power is 1.0 within the beamwidth, falls off by 3 dB at 1.7 * the beamwidth, and is effectively zero at twice the beamwidth. It would not be hard to modify Map::Splat to accept an .ANT file, since that's just a list of field strength at each one-degree increment.

After that, Map::Splat ensures that we have the map data for the region of interest. It uses the data from the NASA SRTM survey, which is available from the U.S. Geological Survey. Each elevation file covers a 1 degree tile; the downloader fetches maps covering a 5 × 5 square centered on the transmitter position. (There are no map files for ocean tiles, so if the tower is near the coast, the downloader will simply fail to download some maps. Presumably there are no customers in that area, and no obstacles either.) The maps get unpacked (from zip files), converted with the “srtm2sdf” tool, and stored in /home/freeside/.splat; future Splat calculations will reuse them rather than re-downloading everything.

There is a higher-resolution version of the SRTM data available, but it's organized on the USGS servers in a way that makes it a little more complicated to download, so we don't use it yet. If you do use it at some point, remember to change the “srtm2sdf” and “splat” command lines to “srtm2sdf-hd” and “splat-hd”.

Map::Splat runs Splat with a command line that tells it where to find all those files, the maximum loss level to consider (equal to db_low), the maximum coverage radius, and an assumed height for each receiver (both of which should be export parameters). Splat's output includes:

* A PPM file containing the image, which will be three colors (white and two shades of red). We pull this into ImageMagick and store it in the “image” property of the Map::Splat object.
* A KML file containing a “LatLonBox” element, which gives the bounding box represented by the PPM. We read this with LibXML, read the coordinates of the bounding box, and store them in the “box” property.

Back in process_generate_coverage, the bounding box fields in tower_sector (south, west, east, north) get filled in from the Map::Splat box, and the png() method is called to convert the image to a usable form (it turns the white area transparent so that the image could be used as a ground overlay, though we don't currently do that).

Then we call Map::Splat::polygonize_json(). The first thing this does is open the image as a GDAL raster data set. It sets the spatial reference (WGS84, GPS coordinates) and the transform matrix (so that the image pixels map onto the correct area of real geography). The output of Splat often contains a lot of noise around the edges (where the signal is close to threshold) so it applies a sieve filter that removes blobs smaller than 200 pixels, then calls GDAL's Polygonize() function to turn the red channel into an OGR in-memory polygon layer. Each area of uniform color is a polygon, tagged with the shade of red that it was.

polygonize_json() goes through the list of known colors (we currently have two, db_low and db_high), calls SetAttributeFilter to select all polygons that color, and makes a single polygon that's the union of them. It outputs a GeoJSON feature collection, containing a feature for each signal level, marked with a “level” property. process_generate_coverage takes each of those features and writes its level and geometry (still as GeoJSON) to a sector_coverage record.

= The tower map =
This exists in Freeside 4 but is greatly improved in version 5.

search/tower-map.html is a Google Map displaying three types of features: towers, services, and coverages. Towers and services are point markers, and use a [https://developers.google.com/maps/documentation/javascript/3.exp/reference#Data.StylingFunction styling function] that merges a base style (“baseMarkerStyle”) with the “style” property of each feature, if it has one. This allows style properties (such as marker color) to be overridden per-feature.

Point markers have click and double-click events. “clickHandler” pops up an info window connected to the feature location. If the feature has a “content” property, then that's the HTML to put in the info window. If it has a “url” property, then we make a jQuery.ajax request for that URL, and load the result into the info window. “dblClickHandler” does exactly the same thing, plus it centers the map on the marker and zooms in.

'''Towers''' are all on a single data layer (“tower_data”) which is created within tower-map.html as the @features array. They're styled as markers with a custom “antenna” icon. The info window content is the tower's name, a link to edit/tower.html, and a list of sectors, with the number of up and down services on each sector. If there's a [#Tower exports tower export] with the export_links hook, the links will also be shown here.

The tower info window also has checkboxes to toggle the tower's services and coverages.

'''Services''' are in a separate data layer for each tower (“tower_svc_data[towernum]”). By default all of these layers are hidden; they're turned on from the checkbox by calling setMap(). Each layer is populated by calling [https://developers.google.com/maps/documentation/javascript/datalayer#load_geojson loadGeoJson()] on search/svc_broadband-json.cgi, passing the towernum. Each broadband service is shown as a point marker with color according to its up/down status (red, green, or gray). The info window loads dynamically from view/svc_broadband-popup.html, and shows the customer name and status (small_custview), service label, and the last known service status and latency.

'''Coverages''' are also in a separate layer for each tower (“tower_coverage_data[towernum]”) and hidden by default. They're polygon features. These are populated, in the same way as service layers, from misc/sector_coverage-json.cgi, which pulls their geometry from sector_coverage. The coverage styling function works like the one for point markers, but sets the fill opacity based on whether the coverage feature has the “low” or “high” property. Coverage layers have no click handlers.

Finally, there's a '''search box'''. This uses the [https://developers.google.com/maps/documentation/javascript/places-autocomplete Google Places API], is attached at the top right as a [https://developers.google.com/maps/documentation/javascript/controls#Custom_Controls map control], and lets the user search for addresses or locations near the current display area, in the usual way of Google Maps. What should happen is that clicking on a point that's not a tower or service location should query the sector_coverage table for coverage polygons containing that point, and pop up a window showing which tower/sector has the best estimated signal strength there.

= Tower exports =
Currently only one of these exists (the one for TowerCoverage.com). The code from tower_sector to deal with Splat should be moved into an export also. CloudRF support should be done as yet another export.

These aren't “exports” in the conventional sense of service-provisioning interfaces<nowiki>; </nowiki>the export hooks are invoked from FS::tower_sector's insert(), replace(), and delete() methods. All part_exports that have “tower_sector” in their 'svc' array will be invoked. The export_insert/replace/delete hooks take the tower_sector as an argument. The insert and replace hooks are called after inserting/replacing the record (like for service exports), so if the export needs to modify the tower_sector record, it should set “local $FS::tower_sector::noexport_hack = 1” to avoid recursion.

There's also an export_links hook, which returns optional links to be included in the tower info popup (see above). This takes the tower_sector as an argument.

The get_antenna_types() hook was created specifically for TowerCoverage, and is used by the Edit Tower UI (specifically in elements/tr-tower_sectors.html). It should return an ordered hashref (Tie::IxHash) of hardware_type.typenum values and labels allowed in the “antenna type” property of a sector. (This is the tower antenna, not the customer's antenna.)

Finally, there are “qual” and “qual_result” hooks for prequalifying service at a location. These both take the FS::qual object as an argument, which gives access to the proposed service location (via qual.locationnum) and sector it's trying to link to (qual.sectornum). qual() is invoked from FS::qual::insert, on whichever export was selected for qualifying service. It needs to return a hashref containing:

* status: 'Q' for qualified, 'D' for declined. This is the most important.
* vendor_qual_id: A transaction number for this qualification, if there is one.
* options: A hashref of additional data describing the qualification. This will be inserted into FS::qual_option.

qual_result() is called to interpret the qualification for display (probably by looking at its options). It returns a hashref containing “pkglist”, a hash of pkgparts and labels for the package definitions that could be sold at that location.

This feature is incomplete: the UI for creating quals doesn't have a way to choose a sector, and the TowerCoverage link path implementation doesn't yet do anything useful with the results.

== The TowerCoverage export ==
TowerCoverage.com provides services for both coverage mapping and link path calculation. FS::part_export::tower_towercoverage is an interface to this service.

TowerCoverage requires the sector antenna to use a known configuration. A list of these configurations is hardcoded in the tower_towercoverage module; the first time an export of this type is created, we create a hardware class named “TowerCoverage.com antenna”, and insert the entire list into hardware_type under this class. hardware_type.title contains the TowerCoverage ID of the antenna type. The export has a get_antenna_types() method to list the allowed types.

Certain coverage parameters need to be configured on the export rather than per sector: the client antenna properties (height, gain, and cable loss), maximum range, signal strength thresholds, and frequency band. Splat and/or CloudRF coverage mapping should store these things as export options also.

The export_insert hook does most of the work. It converts the tower_sector properties to the form expected by the API (including metric units on everything, coercing some data types, and naming everything correctly) and creates an insert_coverage queue job. (export_replace does exactly the same thing.)

insert_coverage runs from the queue and sends all its arguments in a POST request to the CoverageAPI endpoint. What it gets back is an XML document containing the path to the rendered map. We don't do anything with the map itself (since we're not logged into the TowerCoverage website). What we do is parse the map file name to get TowerCoverage's ID for this coverage map, and store that in tower_sector.title. export_links() then uses that ID to return a link to their map.

qual() uses the Link Path API. It builds an argument list similar to the one for coverage mapping (where “Site1” is the tower and “Site2” is the client location), sends it to the LinkPathAPI endpoint, and stores everything that comes back in “options”. This is incomplete.

= Station monitoring =
The tower map shows connected customers as green and unreachable customers as red. How does it know?

“freeside-pingd” is a daemon that attempts to scan all IP addresses listed in svc_broadband and tower_sector records. The '''pingd-interval''' config enables this daemon and sets the polling frequency (or you can set it at the command line with “-i”.)

The addr_status table stores the last known ping status of an address. During each scanning pass, the daemon fetches a list of all addresses that haven't been scanned in (interval) seconds (or at all). It then passes each address to a separate process via FS::Daemon::daemon_fork(), which rate-limits to a sensible number (defined here as “ten”) of child processes. Each child process runs the scan() function.

scan() finds the addr_status record for the address, if there is one, then uses Net::Ping to do a TCP probe of the address, with a fixed 5-second timeout. It updates (or inserts) the addr_status with the ping result (up or down), latency, and timestamp. This could be improved by sending multiple pings to estimate packet loss, sending pings with payload data, keeping more than the most recent result, etc.

After recording the ping result, the child process exits. If there are more addresses in the queue, daemon_fork() will start scanning the next one. After all addresses have been scanned, the daemon determines how long to wait before doing another scan. It calculates the expiration time (timestamp + interval) for each addr_status record and finds the lowest expiration time that's in the future. The daemon sleeps until that time, or at most (interval) seconds (since all scan records will be expired by then).

FS::svc_IP_Mixin (services with IP addresses assigned) implements addr_status() (returns the service's addr_status record) and addr_status_color() (returns the CSS color name that goes with the status). search/svc_broadband-json.cgi uses this to set up the marker style. view/svc_broadband-popup.html additionally shows the latency and timestamp from the status record.

[[Category:Freeside 5]]

Freeside:5:Documentation:Tower mapping

2017-01-23T17:53:24Z

Mark:

Freeside:5:Documentation:Tower mapping

2017-01-23T17:51:47Z

Mark: Created page with "This document describes the tower coverage features on the 5.x branch, which has not yet been released. The tower coverage system works, but further development is planned. Re..."

This document describes the tower coverage features on the 5.x branch, which has not yet been released. The tower coverage system works, but further development is planned. Red text is for notes on future improvements.

= Tower coverage calculation =
The goals of tower coverage calculation are:

* ''Mapping'': estimating a tower's coverage area, for marketing, deployment planning, and regulatory reporting.
* ''Prequalification'': estimating the quality of signal available at a customer site, prior to doing an on-site survey.

These require the same calculation: evaluating a signal strength function at a geographic point, using the tower's transmitter characteristics and a terrain map, and comparing to a minimum specified signal strength:

<math>{P}_{\mathit{RX},\mathit{\min }}\le {P}_{\mathit{TX}}+{G}_{\mathit{TX}}-{L}_{\mathit{path}}+{G}_{\mathit{RX}}</math>

where ''PTX'' is the transmitter power, ''GTX'' and ''GRX'' are the transmitter and receiver antenna gains, and ''Lpath'' is the path loss. For an unobstructed path length ''d ''and a frequency ''f'' , the path loss is:

<math>{L}_{\mathit{path}}=20{\log }_{10}4\mathrm{\pi }df/c\approx 36.6\mathrm{dB}+20{\log }_{10}f-20{\log }_{10}d</math>

where ''f'' is in MHz and ''d'' is in miles. For a partially obstructed path, this is more complicated; note that the path is significantly wider than the physical antenna due to diffraction. Consult your radio engineer for details.

Tower configuration is in '''Configuration / Services / Wireless broadband / Towers'''.

= Tower/sector schema =
A record in “tower” represents a site where broadband equipment is installed, including:

* Latitude and longitude
* Tower elevation

A record in “tower_sector” represents a transmitter and antenna installed at a tower. Normally, each wireless broadband service is linked to a sector. For each sector, Freeside tracks:

* IP address
* Height above ground level in feet
* Antenna direction (azimuth and “downtilt”, i.e. elevation)
* Beam width (horizontal and vertical)
* Minimum signal margin
* Power output, in dBm
* Line loss (in cables and connectors), in dB
* Antenna gain, in dB
* “Low-quality” and “high-quality” signal margins. Note that this represents ''maximum path loss'' in dB, not signal strength as such. For example, if db_low = 100, then for “low quality” service to be available, ''PTX'' + ''GTX'' + ''GRX'' − ''PRX,min'' must be at least 100 dB. It is normal for db_low to be larger than db_high. Possibly these should both be export parameters rather than being set per sector. Allowing more than two levels might also be useful.
* “image”, a PNG graphic of the coverage map
* The geographic bounds of the image: west, south, east, and north.
* hardware_typenum, a foreign key to the hardware_type table.
* title, a string field for storing an external ID.

Coverage maps are stored in “sector_coverage”. A separate record is created for each minimum signal level; the intent is that they can be used as map layers. Each one contains:

* sectornum (tower_sector foreign key)
* db_loss, the level of signal loss represented by this map layer
* geometry, a GeoJSON polygon for the region where the signal loss is at most db_loss.

If you move the database into PostGIS, with tower_sector.image becoming a raster and sector_coverage.geometry becoming a polygon layer, then a simple prequalification looks like:

dbh->selectall_arrayref('SELECT sectornum, db_loss FROM sector_coverage WHERE ST_Contains(sector_coverage.geometry, ST_Point(?, ?)) AND db_loss > ? ORDER BY db_loss DESC', $longitude, $latitude, $max_loss)

= Tower mapping with Splat: process_generate_coverage =
Inserting a new tower_sector will queue a process_generate_coverage job for the sector. Editing the height, antenna position or beamwidth, or signal loss levels will also queue this job (from the replace() method). Changing a tower's coordinates will queue process_generate_coverage() for all sectors on the tower.

CloudRF replaces most of this; supposedly they will provide an ESRI shapefile, which we can pull into OGR and convert into a stack of sector_coverage records.

process_generate_coverage makes a temporary directory and creates a Map::Splat object, passing the tower coordinates, antenna height, azimuth and downtilt, vertical and horizontal beamwidth, and the two signal levels to calculate (db_low and db_high). It calls the calculate() method to start generating the map.

Splat is a standalone program that operates on a set of text files. Map::Splat, our wrapper library for it, writes the input files into the temp directory:

* The tower location file (QTH). This only contains the tower coordinates and antenna height.
* The model parameter file (LRP). This contains the frequency, and several static parameters (in particular, the model's safety margin may need to be adjustable).
* The loss contour file (LCF). This maps signal loss levels (dB) to 24-bit colors. Splat always uses white for areas with “no signal” (that is, above the highest specified loss level) and won't accept a color list containing black, or with more than 32 colors. Map::Splat accepts a list of desired signal levels, and spreads them out in the red channel from #04 to #FC. Here, we only have two signal levels, so they'll be at the endpoints.
* The antenna profiles in the azimuth plane (AZ) and the elevation plane (EL). These start with the antenna orientation (which is applied as a rotation to the entire data set) and then list several angular positions and field strengths (relative to the on-axis field strength). Currently we use a simple approximation where the power is 1.0 within the beamwidth, falls off by 3 dB at 1.7 * the beamwidth, and is effectively zero at twice the beamwidth. It would not be hard to modify Map::Splat to accept an .ANT file, since that's just a list of field strength at each one-degree increment.

After that, Map::Splat ensures that we have the map data for the region of interest. It uses the data from the NASA SRTM survey, which is available from the U.S. Geological Survey. Each elevation file covers a 1 degree tile; the downloader fetches maps covering a 5 × 5 square centered on the transmitter position. (There are no map files for ocean tiles, so if the tower is near the coast, the downloader will simply fail to download some maps. Presumably there are no customers in that area, and no obstacles either.) The maps get unpacked (from zip files), converted with the “srtm2sdf” tool, and stored in /home/freeside/.splat; future Splat calculations will reuse them rather than re-downloading everything.

There is a higher-resolution version of the SRTM data available, but it's organized on the USGS servers in a way that makes it a little more complicated to download, so we don't use it yet. If you do use it at some point, remember to change the “srtm2sdf” and “splat” command lines to “srtm2sdf-hd” and “splat-hd”.

Map::Splat runs Splat with a command line that tells it where to find all those files, the maximum loss level to consider (equal to db_low), the maximum coverage radius, and an assumed height for each receiver (both of which should be export parameters). Splat's output includes:

* A PPM file containing the image, which will be three colors (white and two shades of red). We pull this into ImageMagick and store it in the “image” property of the Map::Splat object.
* A KML file containing a “LatLonBox” element, which gives the bounding box represented by the PPM. We read this with LibXML, read the coordinates of the bounding box, and store them in the “box” property.

Back in process_generate_coverage, the bounding box fields in tower_sector (south, west, east, north) get filled in from the Map::Splat box, and the png() method is called to convert the image to a usable form (it turns the white area transparent so that the image could be used as a ground overlay, though we don't currently do that).

Then we call Map::Splat::polygonize_json(). The first thing this does is open the image as a GDAL raster data set. It sets the spatial reference (WGS84, GPS coordinates) and the transform matrix (so that the image pixels map onto the correct area of real geography). The output of Splat often contains a lot of noise around the edges (where the signal is close to threshold) so it applies a sieve filter that removes blobs smaller than 200 pixels, then calls GDAL's Polygonize() function to turn the red channel into an OGR in-memory polygon layer. Each area of uniform color is a polygon, tagged with the shade of red that it was.

polygonize_json() goes through the list of known colors (we currently have two, db_low and db_high), calls SetAttributeFilter to select all polygons that color, and makes a single polygon that's the union of them. It outputs a GeoJSON feature collection, containing a feature for each signal level, marked with a “level” property. process_generate_coverage takes each of those features and writes its level and geometry (still as GeoJSON) to a sector_coverage record.

= The tower map =
This exists in Freeside 4 but is greatly improved in version 5.

search/tower-map.html is a Google Map displaying three types of features: towers, services, and coverages. Towers and services are point markers, and use a [https://developers.google.com/maps/documentation/javascript/3.exp/reference#Data.StylingFunction styling function] that merges a base style (“baseMarkerStyle”) with the “style” property of each feature, if it has one. This allows style properties (such as marker color) to be overridden per-feature.

Point markers have click and double-click events. “clickHandler” pops up an info window connected to the feature location. If the feature has a “content” property, then that's the HTML to put in the info window. If it has a “url” property, then we make a jQuery.ajax request for that URL, and load the result into the info window. “dblClickHandler” does exactly the same thing, plus it centers the map on the marker and zooms in.

'''Towers''' are all on a single data layer (“tower_data”) which is created within tower-map.html as the @features array. They're styled as markers with a custom “antenna” icon. The info window content is the tower's name, a link to edit/tower.html, and a list of sectors, with the number of up and down services on each sector. If there's a [#Tower exports tower export] with the export_links hook, the links will also be shown here.

The tower info window also has checkboxes to toggle the tower's services and coverages.

'''Services''' are in a separate data layer for each tower (“tower_svc_data[towernum]”). By default all of these layers are hidden; they're turned on from the checkbox by calling setMap(). Each layer is populated by calling [https://developers.google.com/maps/documentation/javascript/datalayer#load_geojson loadGeoJson()] on search/svc_broadband-json.cgi, passing the towernum. Each broadband service is shown as a point marker with color according to its up/down status (red, green, or gray). The info window loads dynamically from view/svc_broadband-popup.html, and shows the customer name and status (small_custview), service label, and the last known service status and latency.

'''Coverages''' are also in a separate layer for each tower (“tower_coverage_data[towernum]”) and hidden by default. They're polygon features. These are populated, in the same way as service layers, from misc/sector_coverage-json.cgi, which pulls their geometry from sector_coverage. The coverage styling function works like the one for point markers, but sets the fill opacity based on whether the coverage feature has the “low” or “high” property. Coverage layers have no click handlers.

Finally, there's a '''search box'''. This uses the [https://developers.google.com/maps/documentation/javascript/places-autocomplete Google Places API], is attached at the top right as a [https://developers.google.com/maps/documentation/javascript/controls#Custom_Controls map control], and lets the user search for addresses or locations near the current display area, in the usual way of Google Maps. What should happen is that clicking on a point that's not a tower or service location should query the sector_coverage table for coverage polygons containing that point, and pop up a window showing which tower/sector has the best estimated signal strength there.

= Tower exports =
Currently only one of these exists (the one for TowerCoverage.com). The code from tower_sector to deal with Splat should be moved into an export also. CloudRF support should be done as yet another export.

These aren't “exports” in the conventional sense of service-provisioning interfaces<nowiki>; </nowiki>the export hooks are invoked from FS::tower_sector's insert(), replace(), and delete() methods. All part_exports that have “tower_sector” in their 'svc' array will be invoked. The export_insert/replace/delete hooks take the tower_sector as an argument. The insert and replace hooks are called after inserting/replacing the record (like for service exports), so if the export needs to modify the tower_sector record, it should set “local $FS::tower_sector::noexport_hack = 1” to avoid recursion.

There's also an export_links hook, which returns optional links to be included in the tower info popup (see above). This takes the tower_sector as an argument.

The get_antenna_types() hook was created specifically for TowerCoverage, and is used by the Edit Tower UI (specifically in elements/tr-tower_sectors.html). It should return an ordered hashref (Tie::IxHash) of hardware_type.typenum values and labels allowed in the “antenna type” property of a sector. (This is the tower antenna, not the customer's antenna.)

Finally, there are “qual” and “qual_result” hooks for prequalifying service at a location. These both take the FS::qual object as an argument, which gives access to the proposed service location (via qual.locationnum) and sector it's trying to link to (qual.sectornum). qual() is invoked from FS::qual::insert, on whichever export was selected for qualifying service. It needs to return a hashref containing:

* status: 'Q' for qualified, 'D' for declined. This is the most important.
* vendor_qual_id: A transaction number for this qualification, if there is one.
* options: A hashref of additional data describing the qualification. This will be inserted into FS::qual_option.

qual_result() is called to interpret the qualification for display (probably by looking at its options). It returns a hashref containing “pkglist”, a hash of pkgparts and labels for the package definitions that could be sold at that location.

This feature is incomplete: the UI for creating quals doesn't have a way to choose a sector, and the TowerCoverage link path implementation doesn't yet do anything useful with the results.

== The TowerCoverage export ==
TowerCoverage.com provides services for both coverage mapping and link path calculation. FS::part_export::tower_towercoverage is an interface to this service.

TowerCoverage requires the sector antenna to use a known configuration. A list of these configurations is hardcoded in the tower_towercoverage module; the first time an export of this type is created, we create a hardware class named “TowerCoverage.com antenna”, and insert the entire list into hardware_type under this class. hardware_type.title contains the TowerCoverage ID of the antenna type. The export has a get_antenna_types() method to list the allowed types.

Certain coverage parameters need to be configured on the export rather than per sector: the client antenna properties (height, gain, and cable loss), maximum range, signal strength thresholds, and frequency band. Splat and/or CloudRF coverage mapping should store these things as export options also.

The export_insert hook does most of the work. It converts the tower_sector properties to the form expected by the API (including metric units on everything, coercing some data types, and naming everything correctly) and creates an insert_coverage queue job. (export_replace does exactly the same thing.)

insert_coverage runs from the queue and sends all its arguments in a POST request to the CoverageAPI endpoint. What it gets back is an XML document containing the path to the rendered map. We don't do anything with the map itself (since we're not logged into the TowerCoverage website). What we do is parse the map file name to get TowerCoverage's ID for this coverage map, and store that in tower_sector.title. export_links() then uses that ID to return a link to their map.

qual() uses the Link Path API. It builds an argument list similar to the one for coverage mapping (where “Site1” is the tower and “Site2” is the client location), sends it to the LinkPathAPI endpoint, and stores everything that comes back in “options”. This is incomplete.

= Station monitoring =
The tower map shows connected customers as green and unreachable customers as red. How does it know?

“freeside-pingd” is a daemon that attempts to scan all IP addresses listed in svc_broadband and tower_sector records. The '''pingd-interval''' config enables this daemon and sets the polling frequency (or you can set it at the command line with “-i”.)

The addr_status table stores the last known ping status of an address. During each scanning pass, the daemon fetches a list of all addresses that haven't been scanned in (interval) seconds (or at all). It then passes each address to a separate process via FS::Daemon::daemon_fork(), which rate-limits to a sensible number (defined here as “ten”) of child processes. Each child process runs the scan() function.

scan() finds the addr_status record for the address, if there is one, then uses Net::Ping to do a TCP probe of the address, with a fixed 5-second timeout. It updates (or inserts) the addr_status with the ping result (up or down), latency, and timestamp. This could be improved by sending multiple pings to estimate packet loss, sending pings with payload data, keeping more than the most recent result, etc.

After recording the ping result, the child process exits. If there are more addresses in the queue, daemon_fork() will start scanning the next one. After all addresses have been scanned, the daemon determines how long to wait before doing another scan. It calculates the expiration time (timestamp + interval) for each addr_status record and finds the lowest expiration time that's in the future. The daemon sleeps until that time, or at most (interval) seconds (since all scan records will be expired by then).

FS::svc_IP_Mixin (services with IP addresses assigned) implements addr_status() (returns the service's addr_status record) and addr_status_color() (returns the CSS color name that goes with the status). search/svc_broadband-json.cgi uses this to set up the marker style. view/svc_broadband-popup.html additionally shows the latency and timestamp from the status record.

Freeside:4:Documentation:AFC taxes

2017-01-16T21:38:09Z

Mark:

Avalara currently has two tax services: the one we call "avalara", which is a general sales-tax services, and the one we call "billsoft" and they call "Avalara for Communications" (AFC).

AFC has a batch-oriented FTP interface that it inherited from Billsoft. This is what we support right now. It also has a nice REST interface which is somewhat experimental. Support for this is planned in the future.

= Setup =

Set the '''tax_data_vendor''' config to "billsoft", and the '''billsoft-company-code''' config to your three-letter company code.

Create an upload target (Configuration / with hostname ftp.billsoft.com, and the login and password provided by Avalara.

Import the data files. In '''Tools / Importing / Tax rates''', select "Billsoft PCodes" and import the "all_adr.txt" file. Then select "Tax products" and import the "transervdesc.txt" file.

Run freeside-tax-location-update. This will assign PCodes for all existing package locations that don't yet have one, based on their zip codes.

Assign a tax product to each package definition. See below about this.

== Setting up tax products ==

A "tax product" is a code assigned to a sale to identify what kind of product it is for tax purposes. For example, "recurring charge for local phone service" is a tax product. Tax products are listed in part_pkg_taxproduct. In the UI, entering a string into the tax product field will open an autocomplete search for tax products containing that string.

A package definition (part_pkg) has a single "taxproductnum" field for the base tax product for the package. In the simple case where a package has only one-time or only recurring charges, and no CDR usage charges, this is the only tax product it needs.
[[File:Editing_afc_taxproduct_1.png]]

The "By usage class" button allows tax products to be set separately for setup, recurring non-usage, and usage by class. The taxproductnums are stored in part_pkg options named "usage_taxproduct_FOO" where FOO is the word 'setup' or 'recur' or an integer (a usage_class.classnum).

See the AFC Telecom Mapping Guidelines and other manuals for details on selecting tax products. For usage-billed phone service, AFC has separate tax product groups (T-codes) for interstate, intrastate, and local services. Local services are typically billed as "Local Exchange charge" (7:5); long-distance charges will be intrastate (2:1), interstate (1:1), or international (1:12) toll charges. Other local exchange charges such as FCC subcriber line / LNP recovery are their own categories. This is all different if you're using VoIP or cellular service.

[[File:Editing_afc_taxproduct_2.png]]

(Local service also has a separate, parallel set of tax products ending in the word "Bundle". This is for service plans that don't break out the local and long-distance charge on the bill; for complicated reasons it's exempt from certain taxes.

There should probably be a switch to turn off sending all the CDRs to AFC and just give them a total amount billed in each usage class.

AFC requires that packages that provide "phone lines" of whatever type declare those. Usually this is the "Lines" tax product (such as local voice lines, 7:21, or VoIP lines, 19:21) but there are other options such as "PBX trunks" and even "extensions" (which would require more flexibility in counting units than we currently have). The "Per-line tax product" in the package definition sets the part_pkg.units_taxproductnum to this field.

Finally, there are "Invoice" tax products (always S-code 43). These do not need to be configured. If ''any'' tax product on the invoice has a T-code for which there's an Invoice tax product then an additional line will be submitted declaring the Invoice. For example, if an invoice includes a charge for local phone service (7:5) and one for VoIP access charges (19:6) then it will be sent with both a 7:43 and a 19:43, declaring it as both a local phone invoice and a VoIP invoice.

= Batch workflow =

freeside-daily runs FS::Cron::tax_batch::process_tax_batch. If the tax engine has the 'batch' flag (which only AFC has, probably ever), it will call transfer_batch() on the tax engine.

Note that if AFC is active and a customer has pending invoices, bill_and_collect will NOT run collect() for that customer (i.e. it won't run post-billing events, such as charging credit cards or mailing out invoices). We'll come back to that.

transfer_batch() calls create_batch() to write the batch into a file in $cache_dir/Billsoft/upload. These files follow a strict naming scheme: company code + %Y%m%d + two-letter sequential ID starting with 'AA'. So on Jan 12 2017, the first batch would be FIS20170112AA.CSV, the second would be FIS20170112AB.CSV, etc. Normally you should only need one batch per day, but if you somehow send another one, transfer_batch will detect that there's already one named *AA.CSV (in $cache_dir/Billsoft/upload) and name the next one AB, and so on.

create_batch() finds all pending invoices and writes transaction lines into the batch. See the next section for details.

It then compresses the CSV file into "FTP.ZIP" and uploads that, then polls the server waiting for a result batch. Currently this has a timeout of one day (the $TIMEOUT variable); that's probably longer than it should be.

The result batch will be named the same thing as the uploaded CSV file, but .ZIP, and with "R" after the company code (so "FIS'''R'''20170112AA.ZIP"). It will be saved into $cache_dir/Billsoft/download. It contains several files describing many stages of processing, but the one we want is the detail report, ending in _dtl.rpt.csv. This is a CSV file containing one line for each tax on each transaction, which is conveniently the same data structure as our allocation table.

batch_import() reads this file line by line and creates a tax line item for each distinct tax name. It will also create tax_rate_location, tax_class, and tax_rate records as necessary to describe the taxes that are being charged.

After processing the file, batch_import() removes the pending flags from all invoices, then queues a job to run FS::cust_main::Billing::collect on each customer who had an invoice processed this way.

= What we send them =

AFC tax products have two parts, a transaction code (for the kind of service that's being sold, like local telephone, VoIP, etc.) and a service code (for the kind of charge that this is, like monthly subscription, toll, installation, etc.). I internally call these "T-codes" and "S-codes". See the various AFC manuals (Local Exchange Mapping, VoIP Mapping, etc.) for details.

AFC expects each possibly-taxable feature of the sale to be declared as a separate transaction line. So for each cust_bill_pkg record on the invoice, we send the following transaction lines:
* Each recurring fee (cust_bill_pkg.recur).
* Each non-recurring fee (cust_bill_pkg.setup).
* The number of phone lines (for per-line surcharges).
* Each CDR (for voip_cdr packages). Yes, this is a large number of records.

In almost all cases, per-line or per-customer) charges such as E911 fees need to be applied once per month, so the package must have a monthly billing cycle.

Fields applying to the whole invoice ('''%bill_properties'''):
* RequestType: Always 'CalcTaxes' for now.
* BillTo... (CountryISO, PCode, ZipCode, ZipP4): the customer's billing address.
* Date: The invoice date (YYYYMMDD).
* CustomerType: The customer's tax status ('B'usiness, 'R'esidential, etc.).
* InvoiceNumber: Freeside invnum (echoed in the result file, used to find this invoice again).
* CustomerNumber: Freeside custnum (echoed in the result file, used for error checking).
* and some sitewide flags describing the service provider's status:
** Facilities: Indicates the provider owns the physical infrastructure used to deliver services.
** Franchise: Indicates that the provider sells service pursuant to a franchise agreement with the jurisdiction.
** Regulated: Indicates that the provider is a regulated public utility.
** BusinessClass: Indicates whether the provider is classified as a CLEC or ILEC.

Fields applying to all line items of a specific package ('''%pkg_properties'''):
* Optional: Freeside billpkgnum (echoed in the result file; the field is just a generic place for a numeric identifier).
* Sale: Whether this package is sold on a wholesale basis (a part_pkg flag, currently not available in any package).

Fields sent with recurring fee or setup fee lines:
* Termination... (CountryISO, PCode, ZipCode, ZipP4): the package's service address.
* TransactionType/ServiceType: The tax product for the recurring or setup fee.
* Charge: The amount of the sale.

Fields sent with CDR lines:
* TransactionType/ServiceType: The tax product for the CDR's usage class.
* Charge: The rated charge for the call.
* Minutes: The call duration in minutes (including fraction).
* OriginationNpaNxx: The source number's prefix.
* TerminationNpaNxx: The destination number's prefix.

Note that if LRN lookup is enabled, the Origination/Termination prefixes will be taken from the resolved LRN numbers.

Fields sent with the "number of phone lines" transaction line:
* Charges: Zero.
* Lines: The number of phone services, from FS::part_pkg::calc_units. This transaction line will be sent only if it's greater than zero.
* TransactionType/ServiceType: The tax product selected with part_pkg.units_taxproductnum. Usually this will be S-code 21 ("Lines") but can be different for certain local exchange services.

For each T-code that was present on the invoice, we'll also send a "per invoice" transaction line with S-code 43. The "Lines" and "Charges" are both zero on this line.

= Processing the result file =

batch_import() reads records line by line and looks at several columns:
* InvoiceNumber and Optional columns contain invnum and billpkgnum. We use these to relate the tax back to a taxable line item.
* TaxTypeID (number) and TaxType (string) are the "kind" of tax, like "Sales Tax" or "Universal Service". We create a tax_class record with these values if there isn't one already.
* CountryISO, State, County, Locality, and PCode identify the tax locality. PCodes are unique and we store them as tax_rate_location.geocode. If there isn't a record with this PCode already, we create one.
* TaxLevelID is the taxing authority: national, state/province, county, city, or unincorporated district. We use this to decide which tax_rate_location field to use in the tax description (for example, "CA Sales Tax" vs. "Los Angeles County Sales Tax").
* TaxAmount is the amount of tax, in dollars.

batch_import() constructs the tax name from the geographic name of the taxing authority (like "US" or "NV" or "Las Vegas") and the TaxType string (like "Sales Tax") and finds or creates a tax_rate record with that name, and geocode = the PCode of the locality.

It then calls add_tax_item(), an internal method that creates a cust_bill_pkg record with the correct itemdesc (= the tax name). If there's already one (from another line item subject to the same-named tax) then add_tax_item increments the amount of that item.

Finally, it creates a cust_bill_pkg_tax_rate_location allocation record linking the tax and taxed line items, the tax_rate and tax_rate_location records, and the amount.

If there are errors processing the batch, we log them and continue. The invoice will remain in the pending state and be resubmitted in the next batch.

All pending invoices should be cleared at the end of daily processing. There should be a report / notification if any remain in the system after that, but there currently isn't.

[[Category:Freeside 4]]

Freeside:4:Documentation:AFC taxes

2017-01-16T21:31:04Z

Mark:

Avalara currently has two tax services: the one we call "avalara", which is a general sales-tax services, and the one we call "billsoft" and they call "Avalara for Communications" (AFC).

AFC has a batch-oriented FTP interface that it inherited from Billsoft. This is what we support right now. It also has a nice REST interface which is somewhat experimental. Support for this is planned in the future.

= Setup =

Set the '''tax_data_vendor''' config to "billsoft", and the '''billsoft-company-code''' config to your three-letter company code.

Create an upload target (Configuration / with hostname ftp.billsoft.com, and the login and password provided by Avalara.

Import the data files. In '''Tools / Importing / Tax rates''', select "Billsoft PCodes" and import the "all_adr.txt" file. Then select "Tax products" and import the "transervdesc.txt" file.

Run freeside-tax-location-update. This will assign PCodes for all existing package locations that don't yet have one, based on their zip codes.

Assign a tax product to each package definition. See below about this.

== Setting up tax products ==

A "tax product" is a code assigned to a sale to identify what kind of product it is for tax purposes. For example, "recurring charge for local phone service" is a tax product. Tax products are listed in part_pkg_taxproduct. In the UI, entering a string into the tax product field will open an autocomplete search for tax products containing that string.

A package definition (part_pkg) has a single "taxproductnum" field for the base tax product for the package. In the simple case where a package has only one-time or only recurring charges, and no CDR usage charges, this is the only tax product it needs.
[[File:Editing_afc_taxproduct_1.png]]

The "By usage class" button allows tax products to be set separately for setup, recurring non-usage, and usage by class. The taxproductnums are stored in part_pkg options named "usage_taxproduct_FOO" where FOO is the word 'setup' or 'recur' or an integer (a usage_class.classnum).

See the AFC Telecom Mapping Guidelines and other manuals for details on selecting tax products. For usage-billed phone service, AFC has separate tax product groups (T-codes) for interstate, intrastate, and local services. Local services are typically billed as "Local Exchange charge" (7:5); long-distance charges will be intrastate (2:1), interstate (1:1), or international (1:12) toll charges. Other local exchange charges such as FCC subcriber line / LNP recovery are their own categories. This is all different if you're using VoIP or cellular service.
[[File:Editing_afc_taxproduct_2.png]]

(Local service also has a separate, parallel set of tax products ending in the word "Bundle". This is for service plans that don't break out the local and long-distance charge on the bill; for complicated reasons it's exempt from certain taxes.

There should probably be a switch to turn off sending all the CDRs to AFC and just give them a total amount billed in each usage class.

AFC requires that packages that provide "phone lines" of whatever type declare those. Usually this is the "Lines" tax product (such as local voice lines, 7:21, or VoIP lines, 19:21) but there are other options such as "PBX trunks" and even "extensions" (which would require more flexibility in counting units than we currently have). The "Per-line tax product" in the package definition sets the part_pkg.units_taxproductnum to this field.

Finally, there are "Invoice" tax products (always S-code 43). These do not need to be configured. If ''any'' tax product on the invoice has a T-code for which there's an Invoice tax product then an additional line will be submitted declaring the Invoice. For example, if an invoice includes a charge for local phone service (7:5) and one for VoIP access charges (19:6) then it will be sent with both a 7:43 and a 19:43, declaring it as both a local phone invoice and a VoIP invoice.

= Batch workflow =

freeside-daily runs FS::Cron::tax_batch::process_tax_batch. If the tax engine has the 'batch' flag (which only AFC has, probably ever), it will call transfer_batch() on the tax engine.

Note that if AFC is active and a customer has pending invoices, bill_and_collect will NOT run collect() for that customer (i.e. it won't run post-billing events, such as charging credit cards or mailing out invoices). We'll come back to that.

transfer_batch() calls create_batch() to write the batch into a file in $cache_dir/Billsoft/upload. These files follow a strict naming scheme: company code + %Y%m%d + two-letter sequential ID starting with 'AA'. So on Jan 12 2017, the first batch would be FIS20170112AA.CSV, the second would be FIS20170112AB.CSV, etc. Normally you should only need one batch per day, but if you somehow send another one, transfer_batch will detect that there's already one named *AA.CSV (in $cache_dir/Billsoft/upload) and name the next one AB, and so on.

create_batch() finds all pending invoices and writes transaction lines into the batch. See the next section for details.

It then compresses the CSV file into "FTP.ZIP" and uploads that, then polls the server waiting for a result batch. Currently this has a timeout of one day (the $TIMEOUT variable); that's probably longer than it should be.

The result batch will be named the same thing as the uploaded CSV file, but .ZIP, and with "R" after the company code (so "FIS'''R'''20170112AA.ZIP"). It will be saved into $cache_dir/Billsoft/download. It contains several files describing many stages of processing, but the one we want is the detail report, ending in _dtl.rpt.csv. This is a CSV file containing one line for each tax on each transaction, which is conveniently the same data structure as our allocation table.

batch_import() reads this file line by line and creates a tax line item for each distinct tax name. It will also create tax_rate_location, tax_class, and tax_rate records as necessary to describe the taxes that are being charged.

After processing the file, batch_import() removes the pending flags from all invoices, then queues a job to run FS::cust_main::Billing::collect on each customer who had an invoice processed this way.

= What we send them =

AFC tax products have two parts, a transaction code (for the kind of service that's being sold, like local telephone, VoIP, etc.) and a service code (for the kind of charge that this is, like monthly subscription, toll, installation, etc.). I internally call these "T-codes" and "S-codes". See the various AFC manuals (Local Exchange Mapping, VoIP Mapping, etc.) for details.

AFC expects each possibly-taxable feature of the sale to be declared as a separate transaction line. So for each cust_bill_pkg record on the invoice, we send the following transaction lines:
* Each recurring fee (cust_bill_pkg.recur).
* Each non-recurring fee (cust_bill_pkg.setup).
* The number of phone lines (for per-line surcharges).
* Each CDR (for voip_cdr packages). Yes, this is a large number of records.

In almost all cases, per-line or per-customer) charges such as E911 fees need to be applied once per month, so the package must have a monthly billing cycle.

Fields applying to the whole invoice ('''%bill_properties'''):
* RequestType: Always 'CalcTaxes' for now.
* BillTo... (CountryISO, PCode, ZipCode, ZipP4): the customer's billing address.
* Date: The invoice date (YYYYMMDD).
* CustomerType: The customer's tax status ('B'usiness, 'R'esidential, etc.).
* InvoiceNumber: Freeside invnum (echoed in the result file, used to find this invoice again).
* CustomerNumber: Freeside custnum (echoed in the result file, used for error checking).
* and some sitewide flags describing the service provider's status:
** Facilities: Indicates the provider owns the physical infrastructure used to deliver services.
** Franchise: Indicates that the provider sells service pursuant to a franchise agreement with the jurisdiction.
** Regulated: Indicates that the provider is a regulated public utility.
** BusinessClass: Indicates whether the provider is classified as a CLEC or ILEC.

Fields applying to all line items of a specific package ('''%pkg_properties'''):
* Optional: Freeside billpkgnum (echoed in the result file; the field is just a generic place for a numeric identifier).
* Sale: Whether this package is sold on a wholesale basis (a part_pkg flag, currently not available in any package).

Fields sent with recurring fee or setup fee lines:
* Termination... (CountryISO, PCode, ZipCode, ZipP4): the package's service address.
* TransactionType/ServiceType: The tax product for the recurring or setup fee.
* Charge: The amount of the sale.

Fields sent with CDR lines:
* TransactionType/ServiceType: The tax product for the CDR's usage class.
* Charge: The rated charge for the call.
* Minutes: The call duration in minutes (including fraction).
* OriginationNpaNxx: The source number's prefix.
* TerminationNpaNxx: The destination number's prefix.

Note that if LRN lookup is enabled, the Origination/Termination prefixes will be taken from the resolved LRN numbers.

Fields sent with the "number of phone lines" transaction line:
* Charges: Zero.
* Lines: The number of phone services, from FS::part_pkg::calc_units. This transaction line will be sent only if it's greater than zero.
* TransactionType/ServiceType: The tax product selected with part_pkg.units_taxproductnum. Usually this will be S-code 21 ("Lines") but can be different for certain local exchange services.

For each T-code that was present on the invoice, we'll also send a "per invoice" transaction line with S-code 43. The "Lines" and "Charges" are both zero on this line.

= Processing the result file =

batch_import() reads records line by line and looks at several columns:
* InvoiceNumber and Optional columns contain invnum and billpkgnum. We use these to relate the tax back to a taxable line item.
* TaxTypeID (number) and TaxType (string) are the "kind" of tax, like "Sales Tax" or "Universal Service". We create a tax_class record with these values if there isn't one already.
* CountryISO, State, County, Locality, and PCode identify the tax locality. PCodes are unique and we store them as tax_rate_location.geocode. If there isn't a record with this PCode already, we create one.
* TaxLevelID is the taxing authority: national, state/province, county, city, or unincorporated district. We use this to decide which tax_rate_location field to use in the tax description (for example, "CA Sales Tax" vs. "Los Angeles County Sales Tax").
* TaxAmount is the amount of tax, in dollars.

batch_import() constructs the tax name from the geographic name of the taxing authority (like "US" or "NV" or "Las Vegas") and the TaxType string (like "Sales Tax") and finds or creates a tax_rate record with that name, and geocode = the PCode of the locality.

It then calls add_tax_item(), an internal method that creates a cust_bill_pkg record with the correct itemdesc (= the tax name). If there's already one (from another line item subject to the same-named tax) then add_tax_item increments the amount of that item.

Finally, it creates a cust_bill_pkg_tax_rate_location allocation record linking the tax and taxed line items, the tax_rate and tax_rate_location records, and the amount.

If there are errors processing the batch, we log them and continue. The invoice will remain in the pending state and be resubmitted in the next batch.

All pending invoices should be cleared at the end of daily processing. There should be a report / notification if any remain in the system after that, but there currently isn't.

[[Category:Freeside 4]]

Freeside:4:Documentation:TaxEngine

2017-01-16T21:30:16Z

Mark:

= Tax Engines =

Freeside 4 refactors the code for tax calculation into "tax engines". This document describes the interface to use a tax engine for tax calculation, or to implement a new tax calculation method.

A tax engine is a method for calculating tax on invoices. Before version 4, Freeside supported two methods:

* "Internal" tax, using the ''cust_main_county'' table to define tax rates manually. Tax elements on each line item are stored in ''cust_bill_pkg_tax_location'' and linked to both ''cust_main_county'' and ''cust_location''.
* "External" tax using tax tables downloaded from CCH. The name is misleading, as Freeside performs the tax calculations internally using tables stored in the Freeside database.

Starting with version 4, "External" is renamed to "CCH", and the following new methods are available:
* [http://www.avalara.com Avalara]
* [[Freeside:4:Documentation:AFC_taxes|Avalara for Communications, AKA Billsoft]]
* [http://www.suretax.com SureTax]

== External interface ==

The ''enable_taxproducts'' config sets the tax engine to use on the system. FS::TaxEngine->new will return an object of that type.

The "normal" workflow is:

* As each line item is created, call ''add_sale'', passing that line item. Not all tax engines use this.
* Insert the invoice and its line items.
* Call ''calculate_taxes'', passing the FS::cust_bill object.
** This calls ''make_taxlines'' to create a list of tax elements.
** It then calls ''consolidate_taxlines'' to group the elements by tax name and create a list of tax line items (FS::cust_bill_pkg objects).
** ''calculate_taxes'' will return that list, with each line item linked to an array of tax elements.
* Insert the tax line items.
* Add the total tax to the invoice's "charged" field, and set the "pending" field to null.

FS::TaxEngine objects also implement a few peripheral methods:

* ''info'': Returns some metadata about the tax engine, such as which tables it uses for tax rates and elements, whether it requires batch submission of invoices, allows manual location selection, etc.
* ''cust_tax_locations'': Takes an FS::cust_location object, returns a list of tax jurisdictions that it could possibly be in. Only used if manual location selection is allowed.
* ''add_taxproduct'': A "taxproduct" (FS::part_pkg_taxproduct) is a category of taxable products. Some tax systems predefine these and provide a large list. Others expect the user to define them; in that case, this method is called when the user tries to manually add a taxproduct to the system.

Finally, each tax engine can provide a UI component (under browse/part_pkg_taxproduct/) to select the taxproduct when editing a package definition. This is necessary because the part_pkg_taxproduct table is used in different ways by different engines. See examples in that directory.

== Implementation ==

'''You must implement either ''make_taxlines'' or ''calculate_taxes''.''' Everything else is optional.

If your calculation method produces pre-consolidated line items (that is, one line for each distinct tax name), it should be implemented as ''calculate_taxes''. This takes an FS::cust_bill object and returns a list of FS::cust_bill_pkg objects for the line items. Each of these should have tax elements (either FS::cust_bill_pkg_tax_location or FS::cust_bill_pkg_tax_rate_location) that reference the "billpkgnum" fields of the line items tax was charged on.

If your calculation method just returns a list of taxes for each taxed line item, it should be implemented as ''make_taxlines''. This also takes an FS::cust_bill object, and returns a list of tax elements. The tax line items will be created from those. In this case, the ''info'' hash will need to include ''link_table'', the name of the table where you store tax elements (e.g. cust_bill_pkg_tax_location).

[[Category:Freeside 4]]

Category:Freeside 4

2017-01-16T21:29:58Z

Mark: Created page with "Pages related to Freeside version 4."

Pages related to Freeside version 4.

Freeside:4:Documentation:TaxEngine

2017-01-16T21:29:23Z

Mark:

Freeside:4:Documentation:AFC taxes

2017-01-16T21:12:53Z

Mark: clarify things

Avalara currently has two tax services: the one we call "avalara", which is a general sales-tax services, and the one we call "billsoft" and they call "Avalara for Communications" (AFC).

AFC has a batch-oriented FTP interface that it inherited from Billsoft. This is what we support right now. It also has a nice REST interface which is somewhat experimental. Support for this is planned in the future.

= Setup =

Set the '''tax_data_vendor''' config to "billsoft", and the '''billsoft-company-code''' config to your three-letter company code.

Create an upload target (Configuration / with hostname ftp.billsoft.com, and the login and password provided by Avalara.

Import the data files. In '''Tools / Importing / Tax rates''', select "Billsoft PCodes" and import the "all_adr.txt" file. Then select "Tax products" and import the "transervdesc.txt" file.

Run freeside-tax-location-update. This will assign PCodes for all existing package locations that don't yet have one, based on their zip codes.

Assign a tax product to each package definition. See below about this.

== Setting up tax products ==

A "tax product" is a code assigned to a sale to identify what kind of product it is for tax purposes. For example, "recurring charge for local phone service" is a tax product. Tax products are listed in part_pkg_taxproduct. In the UI, entering a string into the tax product field will open an autocomplete search for tax products containing that string.

A package definition (part_pkg) has a single "taxproductnum" field for the base tax product for the package. In the simple case where a package has only one-time or only recurring charges, and no CDR usage charges, this is the only tax product it needs.
[[File:Editing_afc_taxproduct_1.png]]

The "By usage class" button allows tax products to be set separately for setup, recurring non-usage, and usage by class. The taxproductnums are stored in part_pkg options named "usage_taxproduct_FOO" where FOO is the word 'setup' or 'recur' or an integer (a usage_class.classnum).

See the AFC Telecom Mapping Guidelines and other manuals for details on selecting tax products. For usage-billed phone service, AFC has separate tax product groups (T-codes) for interstate, intrastate, and local services. Local services are typically billed as "Local Exchange charge" (7:5); long-distance charges will be intrastate (2:1), interstate (1:1), or international (1:12) toll charges. Other local exchange charges such as FCC subcriber line / LNP recovery are their own categories. This is all different if you're using VoIP or cellular service.
[[File:Editing_afc_taxproduct_2.png]]

(Local service also has a separate, parallel set of tax products ending in the word "Bundle". This is for service plans that don't break out the local and long-distance charge on the bill; for complicated reasons it's exempt from certain taxes.

There should probably be a switch to turn off sending all the CDRs to AFC and just give them a total amount billed in each usage class.

AFC requires that packages that provide "phone lines" of whatever type declare those. Usually this is the "Lines" tax product (such as local voice lines, 7:21, or VoIP lines, 19:21) but there are other options such as "PBX trunks" and even "extensions" (which would require more flexibility in counting units than we currently have). The "Per-line tax product" in the package definition sets the part_pkg.units_taxproductnum to this field.

Finally, there are "Invoice" tax products (always S-code 43). These do not need to be configured. If ''any'' tax product on the invoice has a T-code for which there's an Invoice tax product then an additional line will be submitted declaring the Invoice. For example, if an invoice includes a charge for local phone service (7:5) and one for VoIP access charges (19:6) then it will be sent with both a 7:43 and a 19:43, declaring it as both a local phone invoice and a VoIP invoice.

= Batch workflow =

freeside-daily runs FS::Cron::tax_batch::process_tax_batch. If the tax engine has the 'batch' flag (which only AFC has, probably ever), it will call transfer_batch() on the tax engine.

Note that if AFC is active and a customer has pending invoices, bill_and_collect will NOT run collect() for that customer (i.e. it won't run post-billing events, such as charging credit cards or mailing out invoices). We'll come back to that.

transfer_batch() calls create_batch() to write the batch into a file in $cache_dir/Billsoft/upload. These files follow a strict naming scheme: company code + %Y%m%d + two-letter sequential ID starting with 'AA'. So on Jan 12 2017, the first batch would be FIS20170112AA.CSV, the second would be FIS20170112AB.CSV, etc. Normally you should only need one batch per day, but if you somehow send another one, transfer_batch will detect that there's already one named *AA.CSV (in $cache_dir/Billsoft/upload) and name the next one AB, and so on.

create_batch() finds all pending invoices and writes transaction lines into the batch. See the next section for details.

It then compresses the CSV file into "FTP.ZIP" and uploads that, then polls the server waiting for a result batch. Currently this has a timeout of one day (the $TIMEOUT variable); that's probably longer than it should be.

The result batch will be named the same thing as the uploaded CSV file, but .ZIP, and with "R" after the company code (so "FIS'''R'''20170112AA.ZIP"). It will be saved into $cache_dir/Billsoft/download. It contains several files describing many stages of processing, but the one we want is the detail report, ending in _dtl.rpt.csv. This is a CSV file containing one line for each tax on each transaction, which is conveniently the same data structure as our allocation table.

batch_import() reads this file line by line and creates a tax line item for each distinct tax name. It will also create tax_rate_location, tax_class, and tax_rate records as necessary to describe the taxes that are being charged.

After processing the file, batch_import() removes the pending flags from all invoices, then queues a job to run FS::cust_main::Billing::collect on each customer who had an invoice processed this way.

= What we send them =

AFC tax products have two parts, a transaction code (for the kind of service that's being sold, like local telephone, VoIP, etc.) and a service code (for the kind of charge that this is, like monthly subscription, toll, installation, etc.). I internally call these "T-codes" and "S-codes". See the various AFC manuals (Local Exchange Mapping, VoIP Mapping, etc.) for details.

AFC expects each possibly-taxable feature of the sale to be declared as a separate transaction line. So for each cust_bill_pkg record on the invoice, we send the following transaction lines:
* Each recurring fee (cust_bill_pkg.recur).
* Each non-recurring fee (cust_bill_pkg.setup).
* The number of phone lines (for per-line surcharges).
* Each CDR (for voip_cdr packages). Yes, this is a large number of records.

In almost all cases, per-line or per-customer) charges such as E911 fees need to be applied once per month, so the package must have a monthly billing cycle.

Fields applying to the whole invoice ('''%bill_properties'''):
* RequestType: Always 'CalcTaxes' for now.
* BillTo... (CountryISO, PCode, ZipCode, ZipP4): the customer's billing address.
* Date: The invoice date (YYYYMMDD).
* CustomerType: The customer's tax status ('B'usiness, 'R'esidential, etc.).
* InvoiceNumber: Freeside invnum (echoed in the result file, used to find this invoice again).
* CustomerNumber: Freeside custnum (echoed in the result file, used for error checking).
* and some sitewide flags describing the service provider's status:
** Facilities: Indicates the provider owns the physical infrastructure used to deliver services.
** Franchise: Indicates that the provider sells service pursuant to a franchise agreement with the jurisdiction.
** Regulated: Indicates that the provider is a regulated public utility.
** BusinessClass: Indicates whether the provider is classified as a CLEC or ILEC.

Fields applying to all line items of a specific package ('''%pkg_properties'''):
* Optional: Freeside billpkgnum (echoed in the result file; the field is just a generic place for a numeric identifier).
* Sale: Whether this package is sold on a wholesale basis (a part_pkg flag, currently not available in any package).

Fields sent with recurring fee or setup fee lines:
* Termination... (CountryISO, PCode, ZipCode, ZipP4): the package's service address.
* TransactionType/ServiceType: The tax product for the recurring or setup fee.
* Charge: The amount of the sale.

Fields sent with CDR lines:
* TransactionType/ServiceType: The tax product for the CDR's usage class.
* Charge: The rated charge for the call.
* Minutes: The call duration in minutes (including fraction).
* OriginationNpaNxx: The source number's prefix.
* TerminationNpaNxx: The destination number's prefix.

Note that if LRN lookup is enabled, the Origination/Termination prefixes will be taken from the resolved LRN numbers.

Fields sent with the "number of phone lines" transaction line:
* Charges: Zero.
* Lines: The number of phone services, from FS::part_pkg::calc_units. This transaction line will be sent only if it's greater than zero.
* TransactionType/ServiceType: The tax product selected with part_pkg.units_taxproductnum. Usually this will be S-code 21 ("Lines") but can be different for certain local exchange services.

For each T-code that was present on the invoice, we'll also send a "per invoice" transaction line with S-code 43. The "Lines" and "Charges" are both zero on this line.

= Processing the result file =

batch_import() reads records line by line and looks at several columns:
* InvoiceNumber and Optional columns contain invnum and billpkgnum. We use these to relate the tax back to a taxable line item.
* TaxTypeID (number) and TaxType (string) are the "kind" of tax, like "Sales Tax" or "Universal Service". We create a tax_class record with these values if there isn't one already.
* CountryISO, State, County, Locality, and PCode identify the tax locality. PCodes are unique and we store them as tax_rate_location.geocode. If there isn't a record with this PCode already, we create one.
* TaxLevelID is the taxing authority: national, state/province, county, city, or unincorporated district. We use this to decide which tax_rate_location field to use in the tax description (for example, "CA Sales Tax" vs. "Los Angeles County Sales Tax").
* TaxAmount is the amount of tax, in dollars.

batch_import() constructs the tax name from the geographic name of the taxing authority (like "US" or "NV" or "Las Vegas") and the TaxType string (like "Sales Tax") and finds or creates a tax_rate record with that name, and geocode = the PCode of the locality.

It then calls add_tax_item(), an internal method that creates a cust_bill_pkg record with the correct itemdesc (= the tax name). If there's already one (from another line item subject to the same-named tax) then add_tax_item increments the amount of that item.

Finally, it creates a cust_bill_pkg_tax_rate_location allocation record linking the tax and taxed line items, the tax_rate and tax_rate_location records, and the amount.

If there are errors processing the batch, we log them and continue. The invoice will remain in the pending state and be resubmitted in the next batch.

All pending invoices should be cleared at the end of daily processing. There should be a report / notification if any remain in the system after that, but there currently isn't.

Freeside:4:Documentation:AFC taxes

2017-01-16T21:07:02Z

Mark:

Avalara currently has two tax services: the one we call "avalara", which is a general sales-tax services, and the one we call "billsoft" and they call "Avalara for Communications" (AFC).

AFC has a batch-oriented FTP interface that it inherited from Billsoft. This is what we support right now. It also has a nice REST interface which is somewhat experimental. Support for this is planned in the future.

= Setup =

Set the '''tax_data_vendor''' config to "billsoft", and the '''billsoft-company-code''' config to your three-letter company code.

Create an upload target (Configuration / with hostname ftp.billsoft.com, and the login and password provided by Avalara.

Import the data files. In '''Tools / Importing / Tax rates''', select "Billsoft PCodes" and import the "all_adr.txt" file. Then select "Tax products" and import the "transervdesc.txt" file.

Run freeside-tax-location-update. This will assign PCodes for all existing package locations that don't yet have one, based on their zip codes.

Assign a tax product to each package definition. See below about this.

== Setting up tax products ==

A "tax product" is a code assigned to a sale to identify what kind of product it is for tax purposes. For example, "recurring charge for local phone service" is a tax product. Tax products are listed in part_pkg_taxproduct.

A package definition (part_pkg) has a single "taxproductnum" field for the base tax product for the package. In the simple case where a package has only one-time or only recurring charges, and no CDR usage charges, this is the only tax product it needs.
[[File:Editing_afc_taxproduct_1.png]]

More specifically, tax products can be set using part_pkg options named "usage_taxproduct_FOO" where FOO is the word 'setup' or 'recur' or an integer (a usage_class.classnum). The package will then be broken down into setup, recurring non-usage, and usage (separate by usage class), which are taxed separately. See the AFC Telecom Mapping Guidelines and other manuals for details.

For usage-billed phone service, AFC has separate tax product groups (T-codes) for interstate, intrastate, and local services. Local services are typically billed as "Local Exchange charge" (7:5); long-distance charges will be intrastate (2:1), interstate (1:1), or international (1:12) toll charges. Other local exchange charges such as FCC subcriber line / LNP recovery are their own categories. This is all different if you're using VoIP or cellular service.
[[File:Editing_afc_taxproduct_2.png]]

(Local service also has a separate, parallel set of tax products ending in the word "Bundle". This is for service plans that don't break out the local and long-distance charge on the bill; for complicated reasons it's exempt from certain taxes.

There should probably be a switch to turn off sending all the CDRs to AFC and just give them a total amount billed in each usage class.

AFC requires that packages that provide "phone lines" of whatever type declare those. Usually this is the "Lines" taxproduct (such as local voice lines, 7:21, or VoIP lines, 19:21) but there are other options such as "PBX trunks" and even "extensions" (which would require more flexibility in counting units than we currently have). The "Per-line tax product" in the package definition sets the part_pkg.units_taxproductnum to this field.

Finally, there are "Invoice" tax products (always S-code 43). These do not need to be configured. If ''any'' tax product on the invoice has a T-code for which there's an Invoice tax product then an additional line will be submitted declaring the Invoice. For example, if an invoice includes a charge for local phone service (7:5) and one for VoIP access charges (19:6) then it will be sent with both a 7:43 and a 19:43, declaring it as both a local phone invoice and a VoIP invoice.

= Batch workflow =

freeside-daily runs FS::Cron::tax_batch::process_tax_batch. If the tax engine has the 'batch' flag (which only AFC has, probably ever), it will call transfer_batch() on the tax engine.

Note that if AFC is active and a customer has pending invoices, bill_and_collect will NOT run collect() for that customer (i.e. it won't run post-billing events, such as charging credit cards or mailing out invoices). We'll come back to that.

transfer_batch() calls create_batch() to write the batch into a file in $cache_dir/Billsoft/upload. These files follow a strict naming scheme: company code + %Y%m%d + two-letter sequential ID starting with 'AA'. So on Jan 12 2017, the first batch would be FIS20170112AA.CSV, the second would be FIS20170112AB.CSV, etc. Normally you should only need one batch per day, but if you somehow send another one, transfer_batch will detect that there's already one named *AA.CSV (in $cache_dir/Billsoft/upload) and name the next one AB, and so on.

create_batch() finds all pending invoices and writes transaction lines into the batch. See the next section for details.

It then compresses the CSV file into "FTP.ZIP" and uploads that, then polls the server waiting for a result batch. Currently this has a timeout of one day (the $TIMEOUT variable); that's probably longer than it should be.

The result batch will be named the same thing as the uploaded CSV file, but .ZIP, and with "R" after the company code (so "FIS'''R'''20170112AA.ZIP"). It will be saved into $cache_dir/Billsoft/download. It contains several files describing many stages of processing, but the one we want is the detail report, ending in _dtl.rpt.csv. This is a CSV file containing one line for each tax on each transaction, which is conveniently the same data structure as our allocation table.

batch_import() reads this file line by line and creates a tax line item for each distinct tax name. It will also create tax_rate_location, tax_class, and tax_rate records as necessary to describe the taxes that are being charged.

After processing the file, batch_import() removes the pending flags from all invoices, then queues a job to run FS::cust_main::Billing::collect on each customer who had an invoice processed this way.

= What we send them =

AFC tax products have two parts, a transaction code (for the kind of service that's being sold, like local telephone, VoIP, etc.) and a service code (for the kind of charge that this is, like monthly subscription, toll, installation, etc.). I internally call these "T-codes" and "S-codes". See the various AFC manuals (Local Exchange Mapping, VoIP Mapping, etc.) for details.

AFC expects each possibly-taxable feature of the sale to be declared as a separate transaction line. So for each cust_bill_pkg record on the invoice, we send the following transaction lines:
* Each recurring fee (cust_bill_pkg.recur).
* Each non-recurring fee (cust_bill_pkg.setup).
* The number of phone lines (for per-line surcharges).
* Each CDR (for voip_cdr packages). Yes, this is a large number of records.

In almost all cases, per-line or per-customer) charges such as E911 fees need to be applied once per month, so the package must have a monthly billing cycle.

Fields applying to the whole invoice ('''%bill_properties'''):
* RequestType: Always 'CalcTaxes' for now.
* BillTo... (CountryISO, PCode, ZipCode, ZipP4): the customer's billing address.
* Date: The invoice date (YYYYMMDD).
* CustomerType: The customer's tax status ('B'usiness, 'R'esidential, etc.).
* InvoiceNumber: Freeside invnum (echoed in the result file, used to find this invoice again).
* CustomerNumber: Freeside custnum (echoed in the result file, used for error checking).
* and some sitewide flags describing the service provider's status:
** Facilities: Indicates the provider owns the physical infrastructure used to deliver services.
** Franchise: Indicates that the provider sells service pursuant to a franchise agreement with the jurisdiction.
** Regulated: Indicates that the provider is a regulated public utility.
** BusinessClass: Indicates whether the provider is classified as a CLEC or ILEC.

Fields applying to all line items of a specific package ('''%pkg_properties'''):
* Optional: Freeside billpkgnum (echoed in the result file; the field is just a generic place for a numeric identifier).
* Sale: Whether this package is sold on a wholesale basis (a part_pkg flag, currently not available in any package).

Fields sent with recurring fee or setup fee lines:
* Termination... (CountryISO, PCode, ZipCode, ZipP4): the package's service address.
* TransactionType/ServiceType: The tax product for the recurring or setup fee.
* Charge: The amount of the sale.

Fields sent with CDR lines:
* TransactionType/ServiceType: The tax product for the CDR's usage class.
* Charge: The rated charge for the call.
* Minutes: The call duration in minutes (including fraction).
* OriginationNpaNxx: The source number's prefix.
* TerminationNpaNxx: The destination number's prefix.

Note that if LRN lookup is enabled, the Origination/Termination prefixes will be taken from the resolved LRN numbers.

Fields sent with the "number of phone lines" transaction line:
* Charges: Zero.
* Lines: The number of phone services, from FS::part_pkg::calc_units. This transaction line will be sent only if it's greater than zero.
* TransactionType/ServiceType: The tax product selected with part_pkg.units_taxproductnum. Usually this will be S-code 21 ("Lines") but can be different for certain local exchange services.

For each T-code that was present on the invoice, we'll also send a "per invoice" transaction line with S-code 43. The "Lines" and "Charges" are both zero on this line.

= Processing the result file =

batch_import() reads records line by line and looks at several columns:
* InvoiceNumber and Optional columns contain invnum and billpkgnum. We use these to relate the tax back to a taxable line item.
* TaxTypeID (number) and TaxType (string) are the "kind" of tax, like "Sales Tax" or "Universal Service". We create a tax_class record with these values if there isn't one already.
* CountryISO, State, County, Locality, and PCode identify the tax locality. PCodes are unique and we store them as tax_rate_location.geocode. If there isn't a record with this PCode already, we create one.
* TaxLevelID is the taxing authority: national, state/province, county, city, or unincorporated district. We use this to decide which tax_rate_location field to use in the tax description (for example, "CA Sales Tax" vs. "Los Angeles County Sales Tax").
* TaxAmount is the amount of tax, in dollars.

batch_import() constructs the tax name from the geographic name of the taxing authority (like "US" or "NV" or "Las Vegas") and the TaxType string (like "Sales Tax") and finds or creates a tax_rate record with that name, and geocode = the PCode of the locality.

It then calls add_tax_item(), an internal method that creates a cust_bill_pkg record with the correct itemdesc (= the tax name). If there's already one (from another line item subject to the same-named tax) then add_tax_item increments the amount of that item.

Finally, it creates a cust_bill_pkg_tax_rate_location allocation record linking the tax and taxed line items, the tax_rate and tax_rate_location records, and the amount.

If there are errors processing the batch, we log them and continue. The invoice will remain in the pending state and be resubmitted in the next batch.

All pending invoices should be cleared at the end of daily processing. There should be a report / notification if any remain in the system after that, but there currently isn't.

Freeside:4:Documentation:AFC taxes

2017-01-16T21:05:04Z

Mark:

= Avalara for Communications ("Billsoft") =

Avalara currently has two tax services: the one we call "avalara", which is a general sales-tax services, and the one we call "billsoft" and they call "Avalara for Communications" (AFC).

AFC has a batch-oriented FTP interface that it inherited from Billsoft. This is what we support right now. It also has a nice REST interface which is somewhat experimental. Support for this is planned in the future.

== Setup ==

Set the '''tax_data_vendor''' config to "billsoft", and the '''billsoft-company-code''' config to your three-letter company code.

Create an upload target (Configuration / with hostname ftp.billsoft.com, and the login and password provided by Avalara.

Import the data files. In '''Tools / Importing / Tax rates''', select "Billsoft PCodes" and import the "all_adr.txt" file. Then select "Tax products" and import the "transervdesc.txt" file.

Run freeside-tax-location-update. This will assign PCodes for all existing package locations that don't yet have one, based on their zip codes.

Assign a tax product to each package definition. See below about this.

== Batch workflow ==

freeside-daily runs FS::Cron::tax_batch::process_tax_batch. If the tax engine has the 'batch' flag (which only AFC has, probably ever), it will call transfer_batch() on the tax engine.

Note that if AFC is active and a customer has pending invoices, bill_and_collect will NOT run collect() for that customer (i.e. it won't run post-billing events, such as charging credit cards or mailing out invoices). We'll come back to that.

transfer_batch() calls create_batch() to write the batch into a file in $cache_dir/Billsoft/upload. These files follow a strict naming scheme: company code + %Y%m%d + two-letter sequential ID starting with 'AA'. So on Jan 12 2017, the first batch would be FIS20170112AA.CSV, the second would be FIS20170112AB.CSV, etc. Normally you should only need one batch per day, but if you somehow send another one, transfer_batch will detect that there's already one named *AA.CSV (in $cache_dir/Billsoft/upload) and name the next one AB, and so on.

create_batch() finds all pending invoices and writes transaction lines into the batch. See the next section for details.

It then compresses the CSV file into "FTP.ZIP" and uploads that, then polls the server waiting for a result batch. Currently this has a timeout of one day (the $TIMEOUT variable); that's probably longer than it should be.

The result batch will be named the same thing as the uploaded CSV file, but .ZIP, and with "R" after the company code (so "FIS'''R'''20170112AA.ZIP"). It will be saved into $cache_dir/Billsoft/download. It contains several files describing many stages of processing, but the one we want is the detail report, ending in _dtl.rpt.csv. This is a CSV file containing one line for each tax on each transaction, which is conveniently the same data structure as our allocation table.

batch_import() reads this file line by line and creates a tax line item for each distinct tax name. It will also create tax_rate_location, tax_class, and tax_rate records as necessary to describe the taxes that are being charged.

After processing the file, batch_import() removes the pending flags from all invoices, then queues a job to run FS::cust_main::Billing::collect on each customer who had an invoice processed this way.

== What we send them ==

AFC tax products have two parts, a transaction code (for the kind of service that's being sold, like local telephone, VoIP, etc.) and a service code (for the kind of charge that this is, like monthly subscription, toll, installation, etc.). I internally call these "T-codes" and "S-codes". See the various AFC manuals (Local Exchange Mapping, VoIP Mapping, etc.) for details.

AFC expects each possibly-taxable feature of the sale to be declared as a separate transaction line. So for each cust_bill_pkg record on the invoice, we send the following transaction lines:
* Each recurring fee (cust_bill_pkg.recur).
* Each non-recurring fee (cust_bill_pkg.setup).
* The number of phone lines (for per-line surcharges).
* Each CDR (for voip_cdr packages). Yes, this is a large number of records.

In almost all cases, per-line or per-customer) charges such as E911 fees need to be applied once per month, so the package must have a monthly billing cycle.

Fields applying to the whole invoice ('''%bill_properties'''):
* RequestType: Always 'CalcTaxes' for now.
* BillTo... (CountryISO, PCode, ZipCode, ZipP4): the customer's billing address.
* Date: The invoice date (YYYYMMDD).
* CustomerType: The customer's tax status ('B'usiness, 'R'esidential, etc.).
* InvoiceNumber: Freeside invnum (echoed in the result file, used to find this invoice again).
* CustomerNumber: Freeside custnum (echoed in the result file, used for error checking).
* and some sitewide flags describing the service provider's status:
** Facilities: Indicates the provider owns the physical infrastructure used to deliver services.
** Franchise: Indicates that the provider sells service pursuant to a franchise agreement with the jurisdiction.
** Regulated: Indicates that the provider is a regulated public utility.
** BusinessClass: Indicates whether the provider is classified as a CLEC or ILEC.

Fields applying to all line items of a specific package ('''%pkg_properties'''):
* Optional: Freeside billpkgnum (echoed in the result file; the field is just a generic place for a numeric identifier).
* Sale: Whether this package is sold on a wholesale basis (a part_pkg flag, currently not available in any package).

Fields sent with recurring fee or setup fee lines:
* Termination... (CountryISO, PCode, ZipCode, ZipP4): the package's service address.
* TransactionType/ServiceType: The tax product for the recurring or setup fee.
* Charge: The amount of the sale.

Fields sent with CDR lines:
* TransactionType/ServiceType: The tax product for the CDR's usage class.
* Charge: The rated charge for the call.
* Minutes: The call duration in minutes (including fraction).
* OriginationNpaNxx: The source number's prefix.
* TerminationNpaNxx: The destination number's prefix.

Note that if LRN lookup is enabled, the Origination/Termination prefixes will be taken from the resolved LRN numbers.

Fields sent with the "number of phone lines" transaction line:
* Charges: Zero.
* Lines: The number of phone services, from FS::part_pkg::calc_units. This transaction line will be sent only if it's greater than zero.
* TransactionType/ServiceType: The tax product selected with part_pkg.units_taxproductnum. Usually this will be S-code 21 ("Lines") but can be different for certain local exchange services.

For each T-code that was present on the invoice, we'll also send a "per invoice" transaction line with S-code 43.

== Processing the result file ==

batch_import() reads records line by line and looks at several columns:
* InvoiceNumber and Optional columns contain invnum and billpkgnum. We use these to relate the tax back to a taxable line item.
* TaxTypeID (number) and TaxType (string) are the "kind" of tax, like "Sales Tax" or "Universal Service". We create a tax_class record with these values if there isn't one already.
* CountryISO, State, County, Locality, and PCode identify the tax locality. PCodes are unique and we store them as tax_rate_location.geocode. If there isn't a record with this PCode already, we create one.
* TaxLevelID is the taxing authority: national, state/province, county, city, or unincorporated district. We use this to decide which tax_rate_location field to use in the tax description (for example, "CA Sales Tax" vs. "Los Angeles County Sales Tax").
* TaxAmount is the amount of tax, in dollars.

batch_import() constructs the tax name from the geographic name of the taxing authority (like "US" or "NV" or "Las Vegas") and the TaxType string (like "Sales Tax") and finds or creates a tax_rate record with that name, and geocode = the PCode of the locality.

It then calls add_tax_item(), an internal method that creates a cust_bill_pkg record with the correct itemdesc (= the tax name). If there's already one (from another line item subject to the same-named tax) then add_tax_item increments the amount of that item.

Finally, it creates a cust_bill_pkg_tax_rate_location allocation record linking the tax and taxed line items, the tax_rate and tax_rate_location records, and the amount.

If there are errors processing the batch, we log them and continue. The invoice will remain in the pending state and be resubmitted in the next batch.

All pending invoices should be cleared at the end of daily processing. There should be a report / notification if any remain in the system after that, but there currently isn't.

=== Setting up tax products ===

A "tax product" is a code assigned to a sale to identify what kind of product it is for tax purposes. For example, "recurring charge for local phone service" is a tax product. Tax products are listed in part_pkg_taxproduct.

A package definition (part_pkg) has a single "taxproductnum" field for the base tax product for the package. In the simple case where a package has only one-time or only recurring charges, and no CDR usage charges, this is the only tax product it needs.
[[File:Editing_afc_taxproduct_1.png]]

More specifically, tax products can be set using part_pkg options named "usage_taxproduct_FOO" where FOO is the word 'setup' or 'recur' or an integer (a usage_class.classnum). The package will then be broken down into setup, recurring non-usage, and usage (separate by usage class), which are taxed separately. See the AFC Telecom Mapping Guidelines and other manuals for details.

For usage-billed phone service, AFC has separate tax product groups (T-codes) for interstate, intrastate, and local services. Local services are typically billed as "Local Exchange charge" (7:5); long-distance charges will be intrastate (2:1), interstate (1:1), or international (1:12) toll charges. Other local exchange charges such as FCC subcriber line / LNP recovery are their own categories. This is all different if you're using VoIP or cellular service.
[[File:Editing_afc_taxproduct_2.png]]

(Local service also has a separate, parallel set of tax products ending in the word "Bundle". This is for service plans that don't break out the local and long-distance charge on the bill; for complicated reasons it's exempt from certain taxes.

There should probably be a switch to turn off sending all the CDRs to AFC and just give them a total amount billed in each usage class.

AFC requires that packages that provide "phone lines" of whatever type declare those. Usually this is the "Lines" taxproduct (such as local voice lines, 7:21, or VoIP lines, 19:21) but there are other options such as "PBX trunks" and even "extensions" (which would require more flexibility in counting units than we currently have). The "Per-line tax product" in the package definition sets the part_pkg.units_taxproductnum to this field.

Finally, there are "Invoice" tax products (always S-code 43). These do not need to be configured. If ''any'' tax product on the invoice has a T-code for which there's an Invoice tax product then an additional line will be submitted declaring the Invoice. For example, if an invoice includes a charge for local phone service (7:5) and one for VoIP access charges (19:6) then it will be sent with both a 7:43 and a 19:43, declaring it as both a local phone invoice and a VoIP invoice.

File:Editing afc taxproduct 2.png

2017-01-16T21:03:19Z

Mark:

File:Editing afc taxproduct 1.png

2017-01-16T21:03:06Z

Mark:

Freeside:4:Documentation:AFC taxes

2017-01-16T21:01:58Z

Mark: add details on AFC

= Avalara for Communications ("Billsoft") =

Avalara currently has two tax services: the one we call "avalara", which is a general sales-tax services, and the one we call "billsoft" and they call "Avalara for Communications" (AFC).

AFC has a batch-oriented FTP interface that it inherited from Billsoft. This is what we support right now. It also has a nice REST interface which is somewhat experimental. Support for this is planned in the future.

== Setup ==

Set the '''tax_data_vendor''' config to "billsoft", and the '''billsoft-company-code''' config to your three-letter company code.

Create an upload target (Configuration / with hostname ftp.billsoft.com, and the login and password provided by Avalara.

Import the data files. In '''Tools / Importing / Tax rates''', select "Billsoft PCodes" and import the "all_adr.txt" file. Then select "Tax products" and import the "transervdesc.txt" file.

Run freeside-tax-location-update. This will assign PCodes for all existing package locations that don't yet have one, based on their zip codes.

Assign a tax product to each package definition. See below about this.

== Batch workflow ==

freeside-daily runs FS::Cron::tax_batch::process_tax_batch. If the tax engine has the 'batch' flag (which only AFC has, probably ever), it will call transfer_batch() on the tax engine.

Note that if AFC is active and a customer has pending invoices, bill_and_collect will NOT run collect() for that customer (i.e. it won't run post-billing events, such as charging credit cards or mailing out invoices). We'll come back to that.

transfer_batch() calls create_batch() to write the batch into a file in $cache_dir/Billsoft/upload. These files follow a strict naming scheme: company code + %Y%m%d + two-letter sequential ID starting with 'AA'. So on Jan 12 2017, the first batch would be FIS20170112AA.CSV, the second would be FIS20170112AB.CSV, etc. Normally you should only need one batch per day, but if you somehow send another one, transfer_batch will detect that there's already one named *AA.CSV (in $cache_dir/Billsoft/upload) and name the next one AB, and so on.

create_batch() finds all pending invoices and writes transaction lines into the batch. See the next section for details.

It then compresses the CSV file into "FTP.ZIP" and uploads that, then polls the server waiting for a result batch. Currently this has a timeout of one day (the $TIMEOUT variable); that's probably longer than it should be.

The result batch will be named the same thing as the uploaded CSV file, but .ZIP, and with "R" after the company code (so "FIS'''R'''20170112AA.ZIP"). It will be saved into $cache_dir/Billsoft/download. It contains several files describing many stages of processing, but the one we want is the detail report, ending in _dtl.rpt.csv. This is a CSV file containing one line for each tax on each transaction, which is conveniently the same data structure as our allocation table.

batch_import() reads this file line by line and creates a tax line item for each distinct tax name. It will also create tax_rate_location, tax_class, and tax_rate records as necessary to describe the taxes that are being charged.

After processing the file, batch_import() removes the pending flags from all invoices, then queues a job to run FS::cust_main::Billing::collect on each customer who had an invoice processed this way.

== What we send them ==

AFC tax products have two parts, a transaction code (for the kind of service that's being sold, like local telephone, VoIP, etc.) and a service code (for the kind of charge that this is, like monthly subscription, toll, installation, etc.). I internally call these "T-codes" and "S-codes". See the various AFC manuals (Local Exchange Mapping, VoIP Mapping, etc.) for details.

AFC expects each possibly-taxable feature of the sale to be declared as a separate transaction line. So for each cust_bill_pkg record on the invoice, we send the following transaction lines:
* Each recurring fee (cust_bill_pkg.recur).
* Each non-recurring fee (cust_bill_pkg.setup).
* The number of phone lines (for per-line surcharges).
* Each CDR (for voip_cdr packages). Yes, this is a large number of records.

In almost all cases, per-line or per-customer) charges such as E911 fees need to be applied once per month, so the package must have a monthly billing cycle.

Fields applying to the whole invoice ('''%bill_properties'''):
* RequestType: Always 'CalcTaxes' for now.
* BillTo... (CountryISO, PCode, ZipCode, ZipP4): the customer's billing address.
* Date: The invoice date (YYYYMMDD).
* CustomerType: The customer's tax status ('B'usiness, 'R'esidential, etc.).
* InvoiceNumber: Freeside invnum (echoed in the result file, used to find this invoice again).
* CustomerNumber: Freeside custnum (echoed in the result file, used for error checking).
* and some sitewide flags describing the service provider's status:
** Facilities: Indicates the provider owns the physical infrastructure used to deliver services.
** Franchise: Indicates that the provider sells service pursuant to a franchise agreement with the jurisdiction.
** Regulated: Indicates that the provider is a regulated public utility.
** BusinessClass: Indicates whether the provider is classified as a CLEC or ILEC.

Fields applying to all line items of a specific package ('''%pkg_properties'''):
* Optional: Freeside billpkgnum (echoed in the result file; the field is just a generic place for a numeric identifier).
* Sale: Whether this package is sold on a wholesale basis (a part_pkg flag, currently not available in any package).

Fields sent with recurring fee or setup fee lines:
* Termination... (CountryISO, PCode, ZipCode, ZipP4): the package's service address.
* TransactionType/ServiceType: The tax product for the recurring or setup fee.
* Charge: The amount of the sale.

Fields sent with CDR lines:
* TransactionType/ServiceType: The tax product for the CDR's usage class.
* Charge: The rated charge for the call.
* Minutes: The call duration in minutes (including fraction).
* OriginationNpaNxx: The source number's prefix.
* TerminationNpaNxx: The destination number's prefix.

Note that if LRN lookup is enabled, the Origination/Termination prefixes will be taken from the resolved LRN numbers.

Fields sent with the "number of phone lines" transaction line:
* Charges: Zero.
* Lines: The number of phone services, from FS::part_pkg::calc_units. This transaction line will be sent only if it's greater than zero.
* TransactionType/ServiceType: The tax product selected with part_pkg.units_taxproductnum. Usually this will be S-code 21 ("Lines") but can be different for certain local exchange services.

For each T-code that was present on the invoice, we'll also send a "per invoice" transaction line with S-code 43.

== Processing the result file ==

batch_import() reads records line by line and looks at several columns:
* InvoiceNumber and Optional columns contain invnum and billpkgnum. We use these to relate the tax back to a taxable line item.
* TaxTypeID (number) and TaxType (string) are the "kind" of tax, like "Sales Tax" or "Universal Service". We create a tax_class record with these values if there isn't one already.
* CountryISO, State, County, Locality, and PCode identify the tax locality. PCodes are unique and we store them as tax_rate_location.geocode. If there isn't a record with this PCode already, we create one.
* TaxLevelID is the taxing authority: national, state/province, county, city, or unincorporated district. We use this to decide which tax_rate_location field to use in the tax description (for example, "CA Sales Tax" vs. "Los Angeles County Sales Tax").
* TaxAmount is the amount of tax, in dollars.

batch_import() constructs the tax name from the geographic name of the taxing authority (like "US" or "NV" or "Las Vegas") and the TaxType string (like "Sales Tax") and finds or creates a tax_rate record with that name, and geocode = the PCode of the locality.

It then calls add_tax_item(), an internal method that creates a cust_bill_pkg record with the correct itemdesc (= the tax name). If there's already one (from another line item subject to the same-named tax) then add_tax_item increments the amount of that item.

Finally, it creates a cust_bill_pkg_tax_rate_location allocation record linking the tax and taxed line items, the tax_rate and tax_rate_location records, and the amount.

If there are errors processing the batch, we log them and continue. The invoice will remain in the pending state and be resubmitted in the next batch.

All pending invoices should be cleared at the end of daily processing. There should be a report / notification if any remain in the system after that, but there currently isn't.

=== Setting up tax products ===

A "tax product" is a code assigned to a sale to identify what kind of product it is for tax purposes. For example, "recurring charge for local phone service" is a tax product. Tax products are listed in part_pkg_taxproduct.

A package definition (part_pkg) has a single "taxproductnum" field for the base tax product for the package. In the simple case where a package has only one-time or only recurring charges, and no CDR usage charges, this is the only tax product it needs.

More specifically, tax products can be set using part_pkg options named "usage_taxproduct_FOO" where FOO is the word 'setup' or 'recur' or an integer (a usage_class.classnum). The package will then be broken down into setup, recurring non-usage, and usage (separate by usage class), which are taxed separately. See the AFC Telecom Mapping Guidelines and other manuals for details.

For usage-billed phone service, AFC has separate tax product groups (T-codes) for interstate, intrastate, and local services. Local services are typically billed as "Local Exchange charge" (7:5); long-distance charges will be intrastate (2:1), interstate (1:1), or international (1:12) toll charges. Other local exchange charges such as FCC subcriber line / LNP recovery are their own categories. This is all different if you're using VoIP or cellular service.

(Local service also has a separate, parallel set of tax products ending in the word "Bundle". This is for service plans that don't break out the local and long-distance charge on the bill; for complicated reasons it's exempt from certain taxes.

There should probably be a switch to turn off sending all the CDRs to AFC and just give them a total amount billed in each usage class.

AFC requires that packages that provide "phone lines" of whatever type declare those. Usually this is the "Lines" taxproduct (such as local voice lines, 7:21, or VoIP lines, 19:21) but there are other options such as "PBX trunks" and even "extensions" (which would require more flexibility in counting units than we currently have). The "Per-line tax product" in the package definition sets the part_pkg.units_taxproductnum to this field.

Finally, there are "Invoice" tax products (always S-code 43). These do not need to be configured. If ''any'' tax product on the invoice has a T-code for which there's an Invoice tax product then an additional line will be submitted declaring the Invoice. For example, if an invoice includes a charge for local phone service (7:5) and one for VoIP access charges (19:6) then it will be sent with both a 7:43 and a 19:43, declaring it as both a local phone invoice and a VoIP invoice.

Freeside:4:Documentation:TaxEngine

2017-01-16T21:01:51Z

Mark: add details on AFC

Tax engine documentation

2017-01-13T00:27:20Z

Mark:

= Tax calculation in general =

Freeside supports a few mechanisms for calculating taxes. On version 4 and later, these share an internal interface, FS::TaxEngine.

On 3.x (and as far back as 1.7, I think) there are two tax mechanisms, without any kind of common interface. All references to them are hardcoded. The systems are:

* [[Internal_taxes|"Internal" taxes via ''cust_main_county''.]] These require manual setup, and the kinds of taxes that it handles are very limited.
* "CCH" taxes via ''tax_rate''.

On 4.x, we add:
* SureTax (the online version of CCH)
* Avalara's "AvaTax" product
* [[AFC_taxes|Avalara's "AvaTax For Communications" product]], formerly known as "Billsoft".

= Requirements =

For many clients (those not selling regulated voice telecom services, and those with national sales tax such as Australia and Canada), taxes are fairly simple. Each customer is located in some tax jurisdiction, which defines a tax rate. This is multiplied by the sale amount to that customer to obtain the tax amount, which is added to their invoice.

For voice call service, it's a lot more complex. A typical U.S. phone service might require the carrier to collect:

* State, county, and local sales tax, typically on the entire sale amount.
* State universal service fund (USF) and TDD fund taxes, based on either the sale amount or just the toll charges.
* Federal USF and excise tax, as a percentage of interstate long-distance charges.
* E911 fees.

Other complications related to taxes:

* Some customers are exempt from some or all taxes.
* Some packages are exempt from some or all taxes, also.
* Some customers are exempt from some taxes up to some threshold amount. (Called "Texas tax" in the code.)
* In general, crediting a sale that was taxed should "give tax back" (that is, also credit the tax that applied to that sale). Because the credit can be for any amount, we have to track which line items the credit was applied to, and how much was applied to tax or non-tax amounts.
* And crediting a sale that was exempt up to a threshold should "give back" exemption, also.
* We should be able to report on the taxable sales, tax-exempt sales, tax credited, and tax collected in any billing period.

= High-level overview =

"Tax line items" are cust_bill_pkg records that have pkgnum = 0 and feepart = null. They will also only have setup fees, not recurring fees, and may have "itemdesc" set to the name of the tax as it should appear on the invoice.

Starting with Freeside 3.x, every tax line item has "allocation records" linking it to both the sale items that it was charged on, and its tax definition. These have two foreign keys to cust_bill_pkg: "billpkgnum" (the tax charge) and "taxable_billpkgnum" (the taxed sale).

Tax calculation happens as the last step in creating an invoice (FS::cust_main::Billing::bill). It generates a list of tax line items (FS::cust_bill_pkg objects) each linked to a list of tax allocation records.

In Freeside 3.x, this involved calling two methods in FS::cust_main::Billing:
* _handle_taxes: Identify the tax definitions that apply to this invoice, and make a list of all sale line items that are subject to each tax definition. This also finds any exemptions that apply.
* calculate_taxes: For each tax definition, calculate its tax on the set of taxable items it applies to. Create an allocation record for each of them. Then consolidate those records to make a single tax line item for each ''tax name''. Insert those records and the allocation records.

All of this was done before ''any'' of the records were inserted, which involved building a large free-floating data structure with references from tax line items to allocation records to taxable items to exemption records. FS::cust_bill_pkg::insert would then traverse this structure and insert the records, turning in-memory references into foreign keys.

In 4.x this is somewhat simpler: FS::cust_main::Billing inserts the invoice, along with all its package line items, then calls FS::TaxEngine::calculate_taxes to get a list of tax line items. It then inserts all of them and adds the tax amount to cust_bill.charged. This was needed to support batch mode, since the invoice may sit for the rest of the day before being submitted for tax calculation. Before the taxes have been added, the invoice has the 'pending' flag set so that we know it's not finished.

'''tax_data_vendor''' selects the tax service to use. If it's a batch-mode service (i.e. billsoft), freeside-daily calls FS::TaxEngine::transfer_batch, which takes all pending invoices and sends them in for batch calculation, then appends the resulting taxes to the invoices and marks them as non-pending.

= Schema =

== For internal taxes ==

[[File:Internal_tax_schema.png]]

Tax rates are defined in cust_main_county, which has the following information:
* The tax rate ("tax"), as a percentage.
* The tax name.
* Where the tax applies: country, state, county, city, and district. Except country, they can be null.
* What the tax applies to: a "taxclass" string, and "setuptax" and "recurtax" flags indicating that setup / recur charges are exempt. (Yes, the names are confusing.)
* "exempt_amount": the per-customer monthly exemption if any.
* "source": keeps track of which tax rates are automatically maintained. Currently only applies to the Washington State Sales Tax.

Tax allocation records are in cust_bill_pkg_tax_location, and contain:
* Foreign keys linking to the cust_bill_pkg records of both the tax and the taxed sale.
* A foreign key to the cust_main_county record defining the tax.
* The amount of tax.
* "taxtype", always 'FS::cust_main_county' for now.
* "pkgnum" and "locationnum" fields, which are now redundant, since the taxed sale is already linked back to the package.

Exempt sale records are in cust_tax_exempt_pkg. One exempt sale record is created for each tax that the sale was exempted from. They contain:
* The billpkgnum of the sale.
* The cust_main_county record (taxnum) it was exempted from.
* The year and month that the exemption occurred. This is to make monthly exemptions easier to calculate.
* The sale amount that was exempted.
* Several "exempt_" flags indicating why the sale was exempt: because the customer is tax-exempt ("cust") or selectively exempt ("cust_taxname"), or the tax doesn't apply to setup / recur fees, or there's a monthly exemption.
* A foreign key to cust_credit_bill_pkg. This is used for negative exemption records created when a credit is applied to a tax-exempt sale. For "normal" exempt sale records it will be null.

Package tax classes are in the part_pkg.taxclass field. This is a string, not a foreign key, though there's a list of allowed tax classes in part_pkg_taxclass.

== For external/vendor taxes ==

[[File:External_tax_schema.png]]

Most tables related to external taxes have a "data_vendor" field which identifies which tax service vendor the record came from / applies to.

Tax rates are defined in tax_rate. For CCH taxes, all fields in tax_rate are used for the tax calculation. For Suretax / Avalara taxes, we still create a record for each tax name + jurisdiction as the tax service tells us about them, but the other fields are left empty. The essential fields are:
* The tax name.
* data_vendor
* Where the tax applies, in terms of the "geocode" field, which links to tax_rate_location.
* What kind of tax it is: "taxclassnum", a foreign key to tax_class.

The allocation records are in cust_bill_pkg_tax_rate_location and are pretty similar to the ones for internal taxes, except that they also have "taxratelocationnum", a link to tax_rate_location.

External taxes have a more complex system for classifying the taxability of packages. "Tax products" are defined by the vendor, and stored locally in part_pkg_taxproduct:
* "taxproduct" is the vendor's code for this product.
* "description" is how it should look to the Freeside user.

Each part_pkg can then be associated with a tax product via the taxproductnum field. However, a single package can generate several kinds of charges with different tax treatment. So there are also part_pkg options named "usage_taxproductnum_FOO" where FOO = "setup", "recur", or a usage class number. This allows different taxes to apply to recurring and non-recurring fees, and to local, interstate, or international calls.

Tax vendors also typically have their own coding scheme for customer locations. We refer to these codes as "geocode". cust_location.geocode is the geocode of a package location. The geocodes are defined in tax_rate_location, which has these fields:
* data_vendor
* geocode
* country, state, county, city

part_pkg_taxrate relates tax products to the taxes that apply to them in each geocoded region. This is the core of the CCH tax engine and is unused for any other tax system.

File:Internal tax schema.png

2017-01-13T00:21:16Z

Mark: Mark uploaded a new version of "File:Internal tax schema.png"

File:Internal tax schema.png

2017-01-12T23:51:13Z

Mark:

Tax engine documentation

2017-01-12T23:39:44Z

Mark: add schema graphic

= Tax calculation in general =

Freeside supports a few mechanisms for calculating taxes. On version 4 and later, these share an internal interface, FS::TaxEngine.

On 3.x (and as far back as 1.7, I think) there are two tax mechanisms, without any kind of common interface. All references to them are hardcoded. The systems are:

* [[Internal_taxes|"Internal" taxes via ''cust_main_county''.]] These require manual setup, and the kinds of taxes that it handles are very limited.
* "CCH" taxes via ''tax_rate''.

On 4.x, we add:
* SureTax (the online version of CCH)
* Avalara's "AvaTax" product
* [[AFC_taxes|Avalara's "AvaTax For Communications" product]], formerly known as "Billsoft".

= Requirements =

For many clients (those not selling regulated voice telecom services, and those with national sales tax such as Australia and Canada), taxes are fairly simple. Each customer is located in some tax jurisdiction, which defines a tax rate. This is multiplied by the sale amount to that customer to obtain the tax amount, which is added to their invoice.

For voice call service, it's a lot more complex. A typical U.S. phone service might require the carrier to collect:

* State, county, and local sales tax, typically on the entire sale amount.
* State universal service fund (USF) and TDD fund taxes, based on either the sale amount or just the toll charges.
* Federal USF and excise tax, as a percentage of interstate long-distance charges.
* E911 fees.

Other complications related to taxes:

* Some customers are exempt from some or all taxes.
* Some packages are exempt from some or all taxes, also.
* Some customers are exempt from some taxes up to some threshold amount. (Called "Texas tax" in the code.)
* In general, crediting a sale that was taxed should "give tax back" (that is, also credit the tax that applied to that sale). Because the credit can be for any amount, we have to track which line items the credit was applied to, and how much was applied to tax or non-tax amounts.
* And crediting a sale that was exempt up to a threshold should "give back" exemption, also.
* We should be able to report on the taxable sales, tax-exempt sales, tax credited, and tax collected in any billing period.

= High-level overview =

"Tax line items" are cust_bill_pkg records that have pkgnum = 0 and feepart = null. They will also only have setup fees, not recurring fees, and may have "itemdesc" set to the name of the tax as it should appear on the invoice.

Starting with Freeside 3.x, every tax line item has "allocation records" linking it to both the sale items that it was charged on, and its tax definition. These have two foreign keys to cust_bill_pkg: "billpkgnum" (the tax charge) and "taxable_billpkgnum" (the taxed sale).

Tax calculation happens as the last step in creating an invoice (FS::cust_main::Billing::bill). It generates a list of tax line items (FS::cust_bill_pkg objects) each linked to a list of tax allocation records.

In Freeside 3.x, this involved calling two methods in FS::cust_main::Billing:
* _handle_taxes: Identify the tax definitions that apply to this invoice, and make a list of all sale line items that are subject to each tax definition. This also finds any exemptions that apply.
* calculate_taxes: For each tax definition, calculate its tax on the set of taxable items it applies to. Create an allocation record for each of them. Then consolidate those records to make a single tax line item for each ''tax name''. Insert those records and the allocation records.

All of this was done before ''any'' of the records were inserted, which involved building a large free-floating data structure with references from tax line items to allocation records to taxable items to exemption records. FS::cust_bill_pkg::insert would then traverse this structure and insert the records, turning in-memory references into foreign keys.

In 4.x this is somewhat simpler: FS::cust_main::Billing inserts the invoice, along with all its package line items, then calls FS::TaxEngine::calculate_taxes to get a list of tax line items. It then inserts all of them and adds the tax amount to cust_bill.charged. This was needed to support batch mode, since the invoice may sit for the rest of the day before being submitted for tax calculation. Before the taxes have been added, the invoice has the 'pending' flag set so that we know it's not finished.

'''tax_data_vendor''' selects the tax service to use. If it's a batch-mode service (i.e. billsoft), freeside-daily calls FS::TaxEngine::transfer_batch, which takes all pending invoices and sends them in for batch calculation, then appends the resulting taxes to the invoices and marks them as non-pending.

= Schema =

== For internal taxes ==

Tax rates are defined in cust_main_county, which has the following information:
* The tax rate ("tax"), as a percentage.
* The tax name.
* Where the tax applies: country, state, county, city, and district. Except country, they can be null.
* What the tax applies to: a "taxclass" string, and "setuptax" and "recurtax" flags indicating that setup / recur charges are exempt. (Yes, the names are confusing.)
* "exempt_amount": the per-customer monthly exemption if any.
* "source": keeps track of which tax rates are automatically maintained. Currently only applies to the Washington State Sales Tax.

Tax allocation records are in cust_bill_pkg_tax_location, and contain:
* Foreign keys linking to the cust_bill_pkg records of both the tax and the taxed sale.
* A foreign key to the cust_main_county record defining the tax.
* The amount of tax.
* "taxtype", always 'FS::cust_main_county' for now.
* "pkgnum" and "locationnum" fields, which are now redundant, since the taxed sale is already linked back to the package.

Exempt sale records are in cust_tax_exempt_pkg. One exempt sale record is created for each tax that the sale was exempted from. They contain:
* The billpkgnum of the sale.
* The cust_main_county record (taxnum) it was exempted from.
* The year and month that the exemption occurred. This is to make monthly exemptions easier to calculate.
* The sale amount that was exempted.
* Several "exempt_" flags indicating why the sale was exempt: because the customer is tax-exempt ("cust") or selectively exempt ("cust_taxname"), or the tax doesn't apply to setup / recur fees, or there's a monthly exemption.
* A foreign key to cust_credit_bill_pkg. This is used for negative exemption records created when a credit is applied to a tax-exempt sale. For "normal" exempt sale records it will be null.

Package tax classes are in the part_pkg.taxclass field. This is a string, not a foreign key, though there's a list of allowed tax classes in part_pkg_taxclass.

== For external/vendor taxes ==

[[File:External_tax_schema.png]]

Most tables related to external taxes have a "data_vendor" field which identifies which tax service vendor the record came from / applies to.

Tax rates are defined in tax_rate. For CCH taxes, all fields in tax_rate are used for the tax calculation. For Suretax / Avalara taxes, we still create a record for each tax name + jurisdiction as the tax service tells us about them, but the other fields are left empty. The essential fields are:
* The tax name.
* data_vendor
* Where the tax applies, in terms of the "geocode" field, which links to tax_rate_location.
* What kind of tax it is: "taxclassnum", a foreign key to tax_class.

The allocation records are in cust_bill_pkg_tax_rate_location and are pretty similar to the ones for internal taxes, except that they also have "taxratelocationnum", a link to tax_rate_location.

External taxes have a more complex system for classifying the taxability of packages. "Tax products" are defined by the vendor, and stored locally in part_pkg_taxproduct:
* "taxproduct" is the vendor's code for this product.
* "description" is how it should look to the Freeside user.

Each part_pkg can then be associated with a tax product via the taxproductnum field. However, a single package can generate several kinds of charges with different tax treatment. So there are also part_pkg options named "usage_taxproductnum_FOO" where FOO = "setup", "recur", or a usage class number. This allows different taxes to apply to recurring and non-recurring fees, and to local, interstate, or international calls.

Tax vendors also typically have their own coding scheme for customer locations. We refer to these codes as "geocode". cust_location.geocode is the geocode of a package location. The geocodes are defined in tax_rate_location, which has these fields:
* data_vendor
* geocode
* country, state, county, city

part_pkg_taxrate relates tax products to the taxes that apply to them in each geocoded region. This is the core of the CCH tax engine and is unused for any other tax system.

File:External tax schema.png

2017-01-12T23:38:29Z

Mark: Mark uploaded a new version of "File:External tax schema.png": resize

Schema diagram for the tax engine, for external taxes (except CCH)

File:External tax schema.png

2017-01-12T23:37:11Z

Mark: Schema diagram for the tax engine, for external taxes (except CCH)

Schema diagram for the tax engine, for external taxes (except CCH)

Tax engine documentation

2017-01-12T23:34:09Z

Mark: Created page with "= Tax calculation in general = Freeside supports a few mechanisms for calculating taxes. On version 4 and later, these share an internal interface, FS::TaxEngine. On 3.x (an..."

= Tax calculation in general =

Freeside supports a few mechanisms for calculating taxes. On version 4 and later, these share an internal interface, FS::TaxEngine.

On 3.x (and as far back as 1.7, I think) there are two tax mechanisms, without any kind of common interface. All references to them are hardcoded. The systems are:

* [[Internal_taxes|"Internal" taxes via ''cust_main_county''.]] These require manual setup, and the kinds of taxes that it handles are very limited.
* "CCH" taxes via ''tax_rate''.

On 4.x, we add:
* SureTax (the online version of CCH)
* Avalara's "AvaTax" product
* [[AFC_taxes|Avalara's "AvaTax For Communications" product]], formerly known as "Billsoft".

== Requirements ==

For many clients (those not selling regulated voice telecom services, and those with national sales tax such as Australia and Canada), taxes are fairly simple. Each customer is located in some tax jurisdiction, which defines a tax rate. This is multiplied by the sale amount to that customer to obtain the tax amount, which is added to their invoice.

For voice call service, it's a lot more complex. A typical U.S. phone service might require the carrier to collect:

* State, county, and local sales tax, typically on the entire sale amount.
* State universal service fund (USF) and TDD fund taxes, based on either the sale amount or just the toll charges.
* Federal USF and excise tax, as a percentage of interstate long-distance charges.
* E911 fees.

Other complications related to taxes:

* Some customers are exempt from some or all taxes.
* Some packages are exempt from some or all taxes, also.
* Some customers are exempt from some taxes up to some threshold amount. (Called "Texas tax" in the code.)
* In general, crediting a sale that was taxed should "give tax back" (that is, also credit the tax that applied to that sale). Because the credit can be for any amount, we have to track which line items the credit was applied to, and how much was applied to tax or non-tax amounts.
* And crediting a sale that was exempt up to a threshold should "give back" exemption, also.
* We should be able to report on the taxable sales, tax-exempt sales, tax credited, and tax collected in any billing period.

== High-level overview ==

"Tax line items" are cust_bill_pkg records that have pkgnum = 0 and feepart = null. They will also only have setup fees, not recurring fees, and may have "itemdesc" set to the name of the tax as it should appear on the invoice.

Starting with Freeside 3.x, every tax line item has "allocation records" linking it to both the sale items that it was charged on, and its tax definition. These have two foreign keys to cust_bill_pkg: "billpkgnum" (the tax charge) and "taxable_billpkgnum" (the taxed sale).

Tax calculation happens as the last step in creating an invoice (FS::cust_main::Billing::bill). It generates a list of tax line items (FS::cust_bill_pkg objects) each linked to a list of tax allocation records.

In Freeside 3.x, this involved calling two methods in FS::cust_main::Billing:
* _handle_taxes: Identify the tax definitions that apply to this invoice, and make a list of all sale line items that are subject to each tax definition. This also finds any exemptions that apply.
* calculate_taxes: For each tax definition, calculate its tax on the set of taxable items it applies to. Create an allocation record for each of them. Then consolidate those records to make a single tax line item for each ''tax name''. Insert those records and the allocation records.

All of this was done before ''any'' of the records were inserted, which involved building a large free-floating data structure with references from tax line items to allocation records to taxable items to exemption records. FS::cust_bill_pkg::insert would then traverse this structure and insert the records, turning in-memory references into foreign keys.

In 4.x this is somewhat simpler: FS::cust_main::Billing inserts the invoice, along with all its package line items, then calls FS::TaxEngine::calculate_taxes to get a list of tax line items. It then inserts all of them and adds the tax amount to cust_bill.charged. This was needed to support batch mode, since the invoice may sit for the rest of the day before being submitted for tax calculation. Before the taxes have been added, the invoice has the 'pending' flag set so that we know it's not finished.

'''tax_data_vendor''' selects the tax service to use. If it's a batch-mode service (i.e. billsoft), freeside-daily calls FS::TaxEngine::transfer_batch, which takes all pending invoices and sends them in for batch calculation, then appends the resulting taxes to the invoices and marks them as non-pending.

== Schema ==

=== For internal taxes ===

Tax rates are defined in cust_main_county, which has the following information:
* The tax rate ("tax"), as a percentage.
* The tax name.
* Where the tax applies: country, state, county, city, and district. Except country, they can be null.
* What the tax applies to: a "taxclass" string, and "setuptax" and "recurtax" flags indicating that setup / recur charges are exempt. (Yes, the names are confusing.)
* "exempt_amount": the per-customer monthly exemption if any.
* "source": keeps track of which tax rates are automatically maintained. Currently only applies to the Washington State Sales Tax.

Tax allocation records are in cust_bill_pkg_tax_location, and contain:
* Foreign keys linking to the cust_bill_pkg records of both the tax and the taxed sale.
* A foreign key to the cust_main_county record defining the tax.
* The amount of tax.
* "taxtype", always 'FS::cust_main_county' for now.
* "pkgnum" and "locationnum" fields, which are now redundant, since the taxed sale is already linked back to the package.

Exempt sale records are in cust_tax_exempt_pkg. One exempt sale record is created for each tax that the sale was exempted from. They contain:
* The billpkgnum of the sale.
* The cust_main_county record (taxnum) it was exempted from.
* The year and month that the exemption occurred. This is to make monthly exemptions easier to calculate.
* The sale amount that was exempted.
* Several "exempt_" flags indicating why the sale was exempt: because the customer is tax-exempt ("cust") or selectively exempt ("cust_taxname"), or the tax doesn't apply to setup / recur fees, or there's a monthly exemption.
* A foreign key to cust_credit_bill_pkg. This is used for negative exemption records created when a credit is applied to a tax-exempt sale. For "normal" exempt sale records it will be null.

Package tax classes are in the part_pkg.taxclass field. This is a string, not a foreign key, though there's a list of allowed tax classes in part_pkg_taxclass.

=== For external/vendor taxes ===

Most tables related to external taxes have a "data_vendor" field which identifies which tax service vendor the record came from / applies to.

Tax rates are defined in tax_rate. For CCH taxes, all fields in tax_rate are used for the tax calculation. For Suretax / Avalara taxes, we still create a record for each tax name + jurisdiction as the tax service tells us about them, but the other fields are left empty. The essential fields are:
* The tax name.
* data_vendor
* Where the tax applies, in terms of the "geocode" field, which links to tax_rate_location.
* What kind of tax it is: "taxclassnum", a foreign key to tax_class.

The allocation records are in cust_bill_pkg_tax_rate_location and are pretty similar to the ones for internal taxes, except that they also have "taxratelocationnum", a link to tax_rate_location.

External taxes have a more complex system for classifying the taxability of packages. "Tax products" are defined by the vendor, and stored locally in part_pkg_taxproduct:
* "taxproduct" is the vendor's code for this product.
* "description" is how it should look to the Freeside user.

Each part_pkg can then be associated with a tax product via the taxproductnum field. However, a single package can generate several kinds of charges with different tax treatment. So there are also part_pkg options named "usage_taxproductnum_FOO" where FOO = "setup", "recur", or a usage class number. This allows different taxes to apply to recurring and non-recurring fees, and to local, interstate, or international calls.

Tax vendors also typically have their own coding scheme for customer locations. We refer to these codes as "geocode". cust_location.geocode is the geocode of a package location. The geocodes are defined in tax_rate_location, which has these fields:
* data_vendor
* geocode
* country, state, county, city

part_pkg_taxrate relates tax products to the taxes that apply to them in each geocoded region. This is the core of the CCH tax engine and is unused for any other tax system.

Main Page

2017-01-12T23:28:57Z

Mark:

== Freeside ==
=== Versions ===

:'''Maintenance Version:''' 3.91
::released Aug 10th, 2016
::v3.x will be end-of-life after Q4 2017

:'''Current Version:''': 4.1
::released Aug 5, 2016

:'''Future''': 5.x / git master
::2018

=== Turn Key Solutions ===
*[http://www.freeside.biz/freeside/services.html#install Installation]
*[http://www.freeside.biz/freeside/products.html Freeside Appliance]

=== Documentation ===

* [[Freeside:2.3:Documentation|2.3 Documentation]]
* [[Freeside:3:Documentation|3.x Documentation]]
* [[Freeside:4:Documentation|4.x Documentation]]
* [[Invoice_template_documentation|Invoice template documentation]] (3.x and later)

=== Support ===
*[http://www.freeside.biz/freeside/services.html#support Freeside Internet Services Inc.]
*[[Freeside:Support:Consultants | Freeside Consultants]]
*[[Freeside:Support:HelpWanted | Help Wanted]]

===Training===
*[[Training syllabus]]

=== Specs in progress ===
* [[Use Cases]]
* [[Business::FraudDetect]]
* [[Event_Refactor]]
* [[UI_Refactor]]
* [[Website_Refactor]]
* [[Browser_support]]
* [[Test Suite]]

=== Historical ===
* [[Batch_Refactor]]
* [[Broadband_Services_Spec]]
* [[Virtual_to_Real_Fields]]
* [[part_pkg Mixin Refactor]]
* A new [[Version_Control_System]]
* [[RPM_Build_system]]

= WIKI Reference =

[http://meta.wikimedia.org/wiki/Help:Editing How to edit pages (wiki markup, etc)] 
[http://www.mediawiki.org/wiki/Help:Configuration_settings Configuration settings list] 
[http://meta.wikipedia.org/wiki/MediaWiki_User%27s_Guide User's Guide]

[[Sandbox]] ← Use this page to test out editing, [http://www.phrases.org.uk/meanings/225200.html learn the ropes], etc.

Main Page

2017-01-12T23:28:42Z

Mark: add link to invoice template doc

== Freeside ==
=== Versions ===

:'''Maintenance Version:''' 3.91
::released Aug 10th, 2016
::v3.x will be end-of-life after Q4 2017

:'''Current Version:''': 4.1
::released Aug 5, 2016

:'''Future''': 5.x / git master
::2018

=== Turn Key Solutions ===
*[http://www.freeside.biz/freeside/services.html#install Installation]
*[http://www.freeside.biz/freeside/products.html Freeside Appliance]

=== Documentation ===

* [[Freeside:2.3:Documentation|2.3 Documentation]]
* [[Freeside:3:Documentation|3.x Documentation]]
* [[Freeside:4:Documentation|4.x Documentation]]
* [[Invoice_template_documentation]] (3.x and later)

=== Support ===
*[http://www.freeside.biz/freeside/services.html#support Freeside Internet Services Inc.]
*[[Freeside:Support:Consultants | Freeside Consultants]]
*[[Freeside:Support:HelpWanted | Help Wanted]]

===Training===
*[[Training syllabus]]

=== Specs in progress ===
* [[Use Cases]]
* [[Business::FraudDetect]]
* [[Event_Refactor]]
* [[UI_Refactor]]
* [[Website_Refactor]]
* [[Browser_support]]
* [[Test Suite]]

=== Historical ===
* [[Batch_Refactor]]
* [[Broadband_Services_Spec]]
* [[Virtual_to_Real_Fields]]
* [[part_pkg Mixin Refactor]]
* A new [[Version_Control_System]]
* [[RPM_Build_system]]

= WIKI Reference =

[http://meta.wikimedia.org/wiki/Help:Editing How to edit pages (wiki markup, etc)] 
[http://www.mediawiki.org/wiki/Help:Configuration_settings Configuration settings list] 
[http://meta.wikipedia.org/wiki/MediaWiki_User%27s_Guide User's Guide]

[[Sandbox]] ← Use this page to test out editing, [http://www.phrases.org.uk/meanings/225200.html learn the ropes], etc.

Invoice template documentation

2017-01-11T04:28:42Z

Mark: Style cleanup

= General notes =
Templates for PDF invoices are in LaTeX, with Perl inclusions in [@-- --@] blocks. Each block can insert a string at its position by either assigning the string to the $OUT variable, or simply returning a string. For details, see Text::Template.

The main function for generating the LaTeX code is FS::Template_Mixin::print_generic. This is a method of the cust_bill object (and also of quotation objects), and takes several arguments described in the perldoc. Most of what this function does is set up the "%invoice_data" hash, which gets imported into local variable space when evaluating the template.

print_generic() actually constructs several templates: 'notes', 'coupon', 'footer', 'smallfooter', 'watermark', and sometimes 'summary'. Exactly which config variable is loaded for each of these is decided in Template_Mixin; look for the line that starts "my $tc". It's '''invoice_$format$part''' (where $format is either "latex" or "html" and $part is "notes", "coupon", etc.). Each template is evaluated, with %invoice_data as arguments, then put back into %invoice_data as "$notes", "$coupon", etc. The main template contains code fragments to include them.

To further complicate things, all of these elements have locale overrides AND invoice mode overrides. If print_generic() was called with a 'mode' argument, those templates will be fetched from the invoice mode + customer locale. If not, they'll be taken from the system config for the customer's agent and locale.

You can obtain the generated LaTeX code for a particular invoice with ''view/cust_bill-tex.cgi'', which takes all the same parameters as ''view/cust_bill.cgi''.

The template mechanics make it difficult to insert graphics other than the logo. During the rendering process, the content of the logo.eps config is written to a file with a random name, and then that name is passed as the 'logo_file' parameter. However, the \includegraphics command has access to the entire filesystem, so one option is to place static graphics somewhere under /home/freeside.

One useful tool for editing a template is Overleaf ([https://www.overleaf.com/ https://www.overleaf.com/]), which is a realtime LaTeX editor. This will require you to upload the logo EPS file with the correct randomly generated name (or edit the references to it) along with the modified longtable.sty file. Also, since it can't evaluate the Perl blocks, you have to work with the generated code rather than the template itself, and then manually edit your changes back into the template. It's still much easier than the cycle of "edit template, set config, download and examine PDF", especially since it will display LaTeX errors in a semi-useful way.

== Positioning ==
This is not used in the standard template but has been used in custom templates. Often in an invoice there's a need to place something at a known position on the page, like a return address that needs to line up with an envelope window. LaTeX isn't designed to facilitate that but LaTeX can do absolutely anything, so: the "textpos" package.

"textpos" requires you to set up a grid system, like this:

\usepackage[absolute]{textpos}
\setlength{\TPHorizModule}{1in}
\setlength{\TPVertModule}{1in}

The environment for a fixed position on the page is "textblock". The syntax is odd; notice parentheses rather than braces, and numbers without units (because the \setlength commands defined the units).

\begin{textblock}{3.25}(0.625, 0.5)
\footnotesize{
This text is in a box starting 0.625in left and 0.5in down from the page
origin, in a box 3.25in wide. It will fit inside an envelope window.
}
\end{textblock}

The textblock behaves a lot like a minipage, and can contain almost anything; in particular, \includegraphics works. It's positioned after everything else on the page, which means there's no protection against overprinting anything.

== The standard invoice template ==
=== Preamble ===
Set up the document class and paper size, load all packages we need. In general, all \usepackage commands should be at the start of the document.

* "fancyhdr" supplies the \fancyhead and \fancyfoot commands to place left, right, and center headers and footers.
* "lastpage" creates the LastPage reference for displaying "page X of Y" labels.
* "ifthen" provides the \ifthenelse command.
* "array" provides support for custom table column formats.
* "longtable" provides tables that can span multiple pages. We ship a modified longtable.sty, which reserves vertical space on the first page to allow for the coupon. The amount of vertical space is in the "\LTextracouponspace" variable.
* "afterpage" was removed from the template in May 2005 and we can remove the package at some point.
* "multirow" is used to make a table cell in the coupon to hold the return address.
* "bigstrut" appears not to be used.
* "truncate" provides a command to apply a width limit to a line of text, so that anything past that gets replaced with an ellipsis. This is highly encouraged for things like package descriptions that could otherwise overflow their table cells.
* "graphicx" provides the \includegraphics command for inserting the EPS of the logo.
* "inputenc" and "fontenc" make LaTeX play nicely with UTF-8 characters.
* "background" is used to create watermarks (such as are used for some past-due notices, or voided invoices). The \backgroundsetup command takes a TikZ argument list, which is extremely powerful. See "texdoc pgf" for all the horrifying details.

Useful things to add here:

* "textpos": see above.
* "geometry" to manage the margins and headers in a centralized way. It also takes the [showframe] option, which will draw lines to visualize the page layout.
* "xcolor" if you want color.
* "tabularx" and/or "tabulary" add more options for setting table column widths.

\documentclass[letterpaper]{article}

\usepackage{fancyhdr,lastpage,ifthen,array,longtable,afterpage,caption,multirow,bigstrut}
\usepackage[breakwords]{truncate} % to avoid overflowing boxes
\usepackage{graphicx} % required for logo graphic
\usepackage[utf8]{inputenc} % multilanguage support
\usepackage[T1]{fontenc}
[@-- if ( length($watermark) ) {
$OUT .= '
\usepackage{background}
\backgroundsetup{
placement=center,
opacity=0.25,
color=black,
angle=0,
contents=' . $watermark . '
}';
}
''<nowiki>;</nowiki>
--@]

Set up dimensions. Most of these were determined empirically and really ought to use the "geometry" package. A few of the dimensions can be adjusted by config variables: \topmargin, \headsep, \textheight.

See [https://en.wikibooks.org/wiki/LaTeX/Page_Layout LaTeX Page Layout] and [http://mirrors.ctan.org/macros/latex/base/letter.pdf texdoc letter] for how LaTeX thinks a letter-class page is structured. Note that everything is laid out statically from the top of the page. The header starts at (1in + \voffset + \topmargin), and is placed in a box with height \headheight, and then the document body starts at (1in + \voffset + \topmargin + \headheight + \headsep), and then the footer starts (\textheight + \footskip) below that.

If the header doesn't fit within \headheight, LaTeX will complain, and then adjust the headheight, pushing everything else down. There's also not actually any space reserved for the footer; it has to fit into the space left below the body container.

Commands for manipulating length variables: \newlength declares them (and all of these are predeclared in the document stylesheet); \setlength sets them to a value; \addtolength adds to them.

If you're adjusting margins, it can be helpful to add "\usepackage{layout}" to the preamble, and then "\newpage\layout" before the final "\end{document}". This will print a diagram showing the layout boxes and their dimensions.

\LTchunksize is an internal variable in longtable for how many rows to process at a time. There's probably no reason to change it.

\addtolength{\voffset}{-0.0cm} % top margin to top of header
\addtolength{\hoffset}{-0.6cm} % left margin on page
\addtolength{\topmargin}{[@-- defined($topmargin) ? $topmargin : '-1.00cm' --@]}
\setlength{\headheight}{2.0cm} % height of header
\setlength{\headsep}{[@-- defined($headsep) ? $headsep : '1.0cm' --@]}
\setlength{\footskip}{1.0cm} % bottom of footer from bottom of text

%\addtolength{\textwidth}{2.1in} % width of text
\setlength{\textwidth}{19.5cm}
\setlength{\textheight}{[@-- defined($textheight) ? $textheight : '19.5cm' --@]}
\setlength{\oddsidemargin}{-0.9cm} % odd page left margin
\setlength{\evensidemargin}{-0.9cm} % even page left margin

\LTchunksize=40

$coupon is a local variable containing the entire generated content of the [#Coupon_template invoice payment coupon]. If for some reason there isn't a payment coupon (the customer doesn't owe any money, or this is a quotation or statement rather than an invoice) then it's an empty string.

\footrule is the macro used to draw a line at the top of the footer. Here, we redefine it so that it doesn't appear on the first page if there's a payment coupon (because the coupon will make its own line, at a different position).

\extracouponspace is a config setting for how much space to reserve for the coupon. We will "shorten" the first page's content by that much, which moves the starting position of the header up.

\renewcommand{\headrulewidth}{0pt}
\renewcommand{\footrulewidth}{1pt}

\renewcommand{\footrule}{
[@--
$coupon ? '\ifthenelse{\equal{\thepage}{1}}' : ''<nowiki>;</nowiki>''
--@]
{
}
{
\vbox to 0pt{\rule{\headwidth}{\footrulewidth}\vss}
}
}
\newcommand{\extracouponspace}{[@-- defined($extracouponspace) ? $extracouponspace : '2.7in' --@]}

Positioning mailing addresses. In current code I'd recommend using the "textpos" package instead. Also a command here to output the dollar sign (since '$' has special properties in LaTeX). The escaping functions in Template_Mixin.pm know about this, and will substitute \dollar throughout.

% Adjust the inset of the mailing address
\newcommand{\addressinset}[1][]{\hspace{1.0cm}}

% Adjust the inset of the return address and logo
\newcommand{\returninset}[1][]{\hspace{-0.25cm}}

% New command for address lines i.e. skip them if blank
\newcommand{\addressline}[1]{\ifthenelse{\equal{#1}{}}{}{#1\\}}

% Inserts dollar symbol
\newcommand{\dollar}[1][]{\symbol{36}}

=== Headers/footers ===
"fancyhdr" allows separate left, right, and center headers and footers (the first argument to \fancyhead or \fancyfoot). We use left and right headers and a center footer.

These commands are supposed to remove the default "plain" style page footer, but we don't use that page style so it's probably not needed.

% Remove plain style header/footer
\fancypagestyle{plain}{
\fancyhead{}
}
\fancyhf{}

The center footer. On page 1, if there's a coupon, the coupon is the footer. In that case, we move the start position up by \extracouponspace (using \vspace) so that it starts above the normal footer position.

The footer to display below the coupon (or if there isn't one) is configurable as '''invoice_latexfooter'''. The footer to show on subsequent pages is '''invoice_latexsmallfooter'''. By default they're both just the company name.

% Define fancy header/footer for first and subsequent pages
\fancyfoot[C]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
[@--
if ($coupon) {
$OUT .= '\vspace{-\extracouponspace}';
$OUT .= '\rule[0.5em]{\textwidth}{\footrulewidth}\\\\';
$OUT .= $coupon;
$OUT .= '\vspace{'.
(defined($couponfootsep) ? $couponfootsep : '0.2in') .
'}';
}
''<nowiki>;</nowiki>''
--@] [@-- $smallerfooter ? '\scriptsize{' : '\small{' --@]
[@-- $footer --@]
}[@-- $coupon ? '\vspace{\extracouponspace}' : '' --@]
}
{ % ... pages
[@-- $smallerfooter ? '\scriptsize{' : '\small{' --@]
[@-- $smallfooter --@]
}
}
}

The right side footer, showing the page number. "emt()" localizes text to the customer's language and then escapes it for LaTeX. "~" is a non-breaking space character. It's suppressed on the first page due to the coupon.

\fancyfoot[R]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
}
{ % ... pages
\small{\thepage~[@-- emt('of') --@]~\pageref{LastPage}}
}
}

The left side header, containing the return address and logo on the first page (and nothing on later pages). \returninset is just a horizontal adjustment. \makebox{} tells LaTeX to put its contents into a single box. That's probably unnecessary because the only thing in the box is a table, which is already a single box.

\begin{tabular}{ll} creates a two-column table. In the first column is a "minipage", which is just a fixed-width container environment (5.5cm here) with its bottom edge aligned to the table cell it's in. In here we put the evaluated "returnaddress" template.

Tables in LaTeX always use & to separate columns and \\ to separate rows. A row must always contain as many columns as in the column spec; otherwise LaTeX will guess about what you mean and will usually get it wrong. \\ at the end of the last row is optional.

The second column contains the logo graphic. See "texdoc graphicx" for a full explanation of this command; it can do many useful things including scaling and repositioning the graphic. For example, if you want the logo to be scaled to be 0.5 inch tall, you can do this: "\includegraphics[height=0.5in]{[@-- $logo_file --@]}".

\fancyhead[L]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
\returninset
\makebox{
\begin{tabular}{ll}
\begin{minipage}[b]{5.5cm}
[@-- $returnaddress --@]
\end{minipage} &
\includegraphics{[@-- $logo_file --@]}\\
\end{tabular}
}
}
{ % ... pages
%\includegraphics{[@-- $logo_file --@]} % Uncomment if you want the logo on all pages.
}
}

The right side header, containing the invoice date, invnum, and custnum in a three-column table with each column centered ({ccc}). "$no_date" and "$no_number" are both enabled by arguments to print_generic() and used when printing statements.

Note that "$date" is already formatted using time2str_local('long'), so something like "Feb 1st, 2017". The template can call time2str() to format other dates; this is an alias to time2str_local so it will use the customer's language settings and LaTeX-escape the result.

This is followed by a horizontal line spanning the table (\hline), then a row where the middle cell contains the $notice_name (usually "Invoice") in \huge font, in small caps (\textsc). The \rule{0pt}{5ex} creates a vertical strut (a zero-width rule) which increases the row height to 5ex. Then another \hline.

Note that \hline should never be followed by \\ and will cause errors if it is.

On subsequent pages we show a smaller version without the notice name.

\fancyhead[R]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
\begin{tabular}{ccc}
[@-- join(' & ', ( $no_date ? '': emt('Invoice date') ),''
( $no_number ? '': emt('Invoice #') ),''
emt('Customer #')
)
--@]\\
\vspace{0.2cm}
\textbf{[@-- $date --@]} & \textbf{[@-- $invnum --@]} & \textbf{[@-- $custnum --@]} \\\hline
\rule{0pt}{5ex} &~~ \huge{\textsc{[@-- emt($notice_name) --@]}} & \\
\vspace{-0.2cm}
& & \\\hline
\end{tabular}
}
{ % ... pages
\small{
\begin{tabular}{lll}
[@-- join(' & ', emt('Invoice date'), emt('Invoice #'), emt('Customer #') ) --@]\\
\textbf{[@-- $date --@]} & \textbf{[@-- $invnum --@]} & \textbf{[@-- $custnum --@]}\\
\end{tabular}
}
}
}

Enable the fancyhdr page style and set the font family.

\pagestyle{fancy}
%% Font options are:
%% bch Bitsream Charter
%% put Utopia
%% phv Adobe Helvetica
%% pnc New Century Schoolbook
%% ptm Times
%% pcr Courier

\renewcommand{\familydefault}{phv}

=== Line item table macros ===
This section is Freeside-specific. The main invoice line item table is an 8-column longtable. Normally the columns are:

* Package number
* Description (6 columns)
* Price

(The 6 columns are used separately for showing CDRs, if there are any.)

If '''invoice-unitprice''' is on, it's:

* Package number
* Description (4 columns)
* Unit price
* Quantity
* Price

According to that setting, create macros for how many columns and how much space to allocate to the description, and for the headers of the Unit Price and Unit Quantity columns.

% Commands for freeside table header...

\newcommand{\FSdescriptionlength} { [@-- $unitprices ? '8.2cm' : '12.8cm' --@] }
\newcommand{\FSdescriptioncolumncount} { [@-- $unitprices ? '4' : '6' --@] }
\newcommand{\FSunitcolumns}{ [@--
$unitprices
? '\makebox[2.5cm][r]{\textbf{~~' . emt('Unit Price') . '}} &' .
'\makebox[1.4cm]{\textbf{~' . emt('Quantity') . '}} & '
: ''--@] }''

\FShead renders the main table header. Using \makebox inside a column forces it to a minimum width. \multicolumn{N}{X}{ ... } is like COLSPAN in HTML: makes a cell spanning N columns. The second argument is how to align the contents of the column (l, r, or c).

\truncate renders its argument into a space with a maximum width. It will break at word boundaries and insert an ellipsis afterward. Note that if we didn't do this, longtable would either squish the surrounding columns or push them off the page rather than word-wrap the description.

\FSusagehead is the same, but used for usage summary sections that show the number of calls and total minutes. The '''usage_class_summary''' config turns this on.

\newcommand{\FShead}{
\hline
\rule{0pt}{2.5ex}
\makebox[1.4cm]{} &
\multicolumn{\FSdescriptioncolumncount}{l}{
\truncate{\FSdescriptionlength}{\textbf{[@-- emt('Description') --@]}}
} &
\FSunitcolumns
\makebox[1.6cm][r]{\textbf{[@-- emt('Amount') --@]}} \\
\hline
}

\newcommand{\FSusagehead}{
\hline
\rule{0pt}{2.5ex}
\makebox[1.4cm]{} &
\multicolumn{4}{l}{
\truncate{\FSdescriptionlength}{\textbf{[@-- emt('Description') --@]}}
} &
\textbf{~~[@-- emt('Calls') --@]} &
\textbf{~~[@-- emt('Duration') --@]} &
\textbf{~~[@-- emt('Amount') --@]} \\
\hline
}

Next, the commands to actually display line items.

\FSdesc will be called once for each element in the @detail_items array. This takes five arguments (that's the [5] in the declaration) which are the values to put in the pkgnum, description, unit price, quantity, and price columns. If unit prices are off then arguments 3 and 4 will be empty.

\FSextdesc will be called once for each element in the 'ext_description' element of the detail item. These contain things like service labels, discount details, prorate details, and manually created package details. This macro takes a single argument and places it in a big multicolumn below the description, in small font.

\FScalldetail is used only for call details. These are preprocessed by FS::TemplateItem_Mixin::details() into LaTeX table rows (pre-escaped, delimited with &). So, the macro just skips the first column and outputs its argument.

\FStotaldesc is used for subtotal and total rows; it puts its first argument in a big multicolumn in the description space, and the second in the amount column.

Finally, \FSusagedesc is for usage summary sections, and puts the total calls, duration, and amount in the correct columns.

% ...description...
\newcommand{\FSdesc}[5]{
\multicolumn{1}{c}{\rule{0pt}{2.5ex}\textbf{#1}} &
\multicolumn{[@-- $unitprices ? '4' : '6' --@]}{l}{
\truncate{\FSdescriptionlength}{\textbf{#2}}
} &
[@-- $unitprices ? ' \multicolumn{1}{r}{\textbf{#3}} &'."\n".
' \multicolumn{1}{r}{\textbf{#4}} &'."\n"
:
--@]
\multicolumn{1}{r}{\textbf{#5}}\\
}
% ...extended description...
\newcommand{\FSextdesc}[1]{
\multicolumn{1}{l}{\rule{0pt}{1.0ex}} &
\multicolumn{6}{l}{
\truncate{12.8cm}{\small{[[User:Mark|Mark]] ([[User talk:Mark|talk]])<nowiki>#1}}</nowiki>
} \\
}
% ...call detail (multiple columns already)...
\newcommand{\FScalldetail}[1]{
\multicolumn{1}{l}{\rule{0pt}{1.0ex}} &
[[User:Mark|Mark]] ([[User talk:Mark|talk]])<nowiki>#1</nowiki>
\\
}
}
% ...and total line items (which use the full 12.8cm length, ignoring
% unitprice/quantity
\newcommand{\FStotaldesc}[2]{
& \multicolumn{6}{l}{
\truncate{12.8cm}{#1}
} & #2\\
}
% ...usage class summary
\newcommand{\FSusagedesc}[4]{
\multicolumn{1}{c}{\rule{0pt}{2.5ex}} &
\multicolumn{4}{l}{\textbf{#1}} &
\multicolumn{1}{r}{\textbf{#2}} &
\multicolumn{1}{r}{\textbf{#3}} &
\multicolumn{1}{r}{\textbf{#4}}
\\
}

=== Main document ===
First, make a minipage for the customer's name and address. This starts at \addressinset + 5 mm (not sure why) and can be up to 7 cm wide. The \makebox does nothing (as above, a minipage is already a single box). The \addressline macro makes a line break only if the argument is non-empty, to avoid errors for breaking an empty line.

\begin{document}
% Headers and footers defined for the first page
\addressinset \rule{0.5cm}{0cm}
\makebox{
\begin{minipage}[t]{7.0cm}
\vspace{[@-- defined($addresssep) ? $addresssep : '0.25cm' --@]}
\textbf{[@-- $payname --@]}\\
\addressline{[@-- $company --@]}
\addressline{[@-- $address1 --@]}
\addressline{[@-- $address2 --@]}
\addressline{[@-- $city --@], [@-- $state --@]~~[@-- $zip --@]}
\addressline{[@-- $country --@]}
\end{minipage}}

Make another minipage for the service address, terms, and PO number. $ship_enable is true if the global '''invoice-ship_address''' or per-customer "invoice_ship_address" flag is on.

The initial \hfill spreads out the two minipages so that they go all the way to the margins. The 'terms' and 'po_line' should probably be in \addressline{} macros but currently aren't.

After that, skip some vertical space. (If you need more vertical space for an invoice format, the 1.5cm here can be cut.)

\hfill
\makebox{
\begin{minipage}[t]{6.4cm}
[@--
if ($ship_enable) {
$OUT .= '\textbf{' . emt('Service Address') . '}\\\\';
$OUT .= "\\addressline{$ship_company}";
$OUT .= "\\addressline{$ship_address1}";
$OUT .= "\\addressline{$ship_address2}";
$OUT .= "\\addressline{$ship_city, $ship_state~~$ship_zip}";
$OUT .= "\\addressline{$ship_country}";
$OUT .= '~\\\\';
}else{
$OUT .= ''<nowiki>;</nowiki>''
}
--@]
\begin{flushright}
[@-- $terms ? emt('Terms') . ': ' . emt($terms) : ''--@]\\''
[@-- $po_line --@]\\
\end{flushright}
\end{minipage}}
\vspace{1.5cm}

Insert the [http://www.freeside.biz/mediawiki/index.php?title=Summary&action=edit&redlink=1 Summary] if there is one. The last thing in the summary template is a page break.

%
[@-- $summary --@]
%

=== Sections ===
Start creating sections, according to the @sections array. Section entries contain the following keys:

* description
* category (pkg_category.categoryname, for package sections by category)
* location (hashref of location fields, for package sections by location)
* pretotal (formatted with \FStotaldesc, before detail lines)
* subtotal (formatted with \FStotaldesc, at the end)
* posttotal (line to display after the end of the section; used for "Balance Due" message and a few other things)
* sort_weight (sort order, but also does complicated things)

and optionally some flags:

* post_total (put the section at the end of the invoice)
* summarized (omit the section entirely)
* usage_section (use \FSusagedesc instead of \FSdesc, see above)
* no_subtotal (omit the subtotal line)

If '''invoice_sections''' is enabled: the $multisection flag in print_generic() is turned on, and @sections is filled by calling FS::Template_Mixin::_items_sections, which makes a section for each package category. Line items for packages go into the package category's section. '''invoice_sections_method''' can be used to divide sections by location rather than by category. In addition, sections are created for previous invoices, taxes, and "adjustments" (credits and payments to this invoice).

If not: a single section is created, with empty description, containing previous invoices, current charges, and adjustments (after the total). '''previous_balance-section''' will make a previous invoices section anyway.

Additional sections are created in some cases. '''svc_phone_sections''' creates a section for each phone line (see RT#6592; note that this customer is no longer active). '''voip-cust_accountcode_cdr''' creates a section for each accountcode (see RT#12181). '''discount-show_available''' creates a section with information on available discounts. '''usage_class_summary''' creates a section with usage subtotals.

'''finance_pkgclass''' specifies the package class containing "finance charges", which is handled specially. That class's category name is passed to the template as $finance_section, and the sum of charges in that category is passed as $finance_amount. That amount is broken out from the "Current charges" line in the summary section. On a multisection summary-format invoice, the finance charges will also ''not'' be shown in the body of the invoice. I don't know why.

Loop over all sections (except for the finance charges, maybe). Inside the loop we'll do everything by appending to $OUT.

\section*{}
[@--
foreach my $section ( grep { !$summary || $_->{description} ne $finance_section } @sections ) {

We're going to do many things if "!$summary". If there's a summary section, the pretotals and posttotals are redundant so skip them.

if ($section->{'pretotal'} && !$summary) {
$OUT .= '\begin{flushright}';
$OUT .= '\large\textsc{'. $section->{'pretotal'}. '}\\\\';
$OUT .= '\\end{flushright}';
}

Start a new page if this is a "post_total" section. As far as I know these are used only for usage details. I think the 'summarized' flag is no longer used for anything.

The section heading is going to be a caption placed inside the main table. \captionsetup sets properties for the caption (left justified, bold, small caps, Large size). If this is the first page, also tell longtable (via \LTextracouponspace) to shorten the effective length of the page so that it doesn't overprint the coupon.

$OUT .= '\pagebreak' if $section->{'post_total'};
unless ($section->{'summarized'} ) {
$OUT .= '\captionsetup{singlelinecheck=false,justification=raggedright,font={Large,sc,bf}}';
$OUT .= '\ifthenelse{\equal{\thepage}{1}}{\setlength{\LTextracouponspace}{\extracouponspace}}{\setlength{\LTextracouponspace}{0pt}}'
if $coupon;

Start the longtable that will contain the section's line items. Eight columns, pkgnum (centered), description and optional unitprice/quantity (left), then price (right).

\caption* produces a table caption without prefixing it with "Table 1". The caption is the section description, or 'Charges' if it's null (except that if packages are sectioned by location, we put one together from the location fields).

$OUT .= '\begin{longtable}{cllllllr}';
$OUT .= '\caption*{ ';
if ($section->{'location'}) {
$OUT .= $section->{'location'}{'label_prefix'}. ': '
if length($section->{'location'}{'label_prefix'});
$OUT .= $section->{'location'}{'address1'};
$OUT .= ', ' . $section->{'location'}{'address2'}
if length($section->{'location'}{'address2'});
$OUT .= ', ' .
$section->{'location'}{'city'} . ', ' .
$section->{'location'}{'state'} . '~' .
$section->{'location'}{'zip'};
} elsif ( $section->{'description'} ) {
$OUT .= ($section->{'description'});
} else {
$OUT .= emt('Charges');
}
$OUT .= '}\\\\';

The point of a longtable is that it can span multiple pages, including printing head/foot rows at the top/bottom of each page automatically. The protocol for setting these up is to specify the first head, per-page head, per-page foot, and last foot, then the rest of the table contents. Here, we set the first head to be \FShead (or \FSusagehead). \endfirsthead means we're done specifying it.

"header_generator" and other foo_generators are from an old mechanism that injected callbacks into the template data. This is strongly deprecated.

if ($section->{header_generator}) {
$OUT .= &{$section->{header_generator}}();
} elsif ( $section->{usage_section} ) {
$OUT .= '\FSusagehead';
} else {
$OUT .= '\FShead';
}
$OUT .= '\endfirsthead';

The per-page head: a row with a "Continued" message, then \FShead again. The per-page foot is the same.

$OUT .= '\multicolumn{7}{r}{\rule{0pt}{2.5ex}'.emt('Continued from previous page').'}\\\\';
if ($section->{header_generator}) {
$OUT .= &{$section->{header_generator}}();
} elsif ( $section->{usage_section} ) {
$OUT .= '\FSusagehead';
} else {
$OUT .= '\FShead';
}
$OUT .= '\endhead';
$OUT .= '\multicolumn{7}{r}{\rule{0pt}{2.5ex}'.emt('Continued on next page...').'}\\\\';
$OUT .= '\endfoot';
$OUT .= '\hline';

The final foot for the section. For multisection invoices this contains the section subtotal:

if (scalar(@sections) > 1 and !$section->{no_subtotal}) {
if ($section->{total_generator}) {
$OUT .= &{$section->{total_generator}}($section);
} else {
$OUT .= '\FStotaldesc{' . $section->{'description'} . ' Total}' .
'{' . $section->{'subtotal'} . '}' . "\n";
}
}

Single-section invoices use "@total_items" for items that need to appear after the line items. These include the pre-tax subtotal, taxes, total charges, adjustments, and the balance due. Call \FStotaldesc for each of them. Then end the section foot.

<nowiki>#if ($section == $sections[$#sections]) {</nowiki>
foreach my $line (grep {$_->{section}->{description} eq $section->{description}} @total_items) {
if ($section->{total_line_generator}) {
$OUT .= &{$section->{total_line_generator}}($line);
} else {
$OUT .= '\FStotaldesc{' . $line->{'total_item'} . '}' .
'{' . $line->{'total_amount'} . '}' . "\n";
}
}
<nowiki>#}</nowiki>
$OUT .= '\hline';
$OUT .= '\endlastfoot';

Line items are passed in @detail_items. A line item entry can contain:

* section (reference)
* ref (pkgnum)
* description
* ext_description (arrayref of additional lines)
* quantity
* amount
* duration (for usage summary sections)
* pkgpart
* unit_amount (unit price)
* locationnum
* svc_label (the primary service's label)

Line items are created in several places in Template_Mixin and TemplateItem_Mixin, most importantly _items_cust_bill_pkg.

@detail_items is a single array; we find the lines belonging to this section by checking their 'section' element. If there's only one section, process all the lines.

my $lastref = 0;
foreach my $line (
grep { ( scalar( @sections ) > 1
? $section->{'description'} eq $_->{'section'}->{'description'}
: 1
) }
@detail_items )
{
my $ext_description = $line->{'ext_description'};

<nowiki># Don't break-up small packages.</nowiki>
my $rowbreak = @$ext_description < 5 ? '*' : ''<nowiki>;</nowiki>''

$lastref is set to the pkgnum of the previous line item. If the value of 'ref' no longer equals that, we're now showing a different package; separate with an \hline. After outputting this line item we'll set $lastref to the new pkgnum.

If this is a usage summary section, convert the duration to minutes/seconds and call \FSusagedesc, passing the description, quantity (number of calls), duration, and total price.

If it's a normal section, call \FSdesc, passing the description, unit price, quantity, and amount. If unit prices aren't in use, or there isn't one on this line item, then leave unit price and quantity blank.

"$rowbreak" is a flag for whether to allow a page break. In general we want to keep line items and their ext_description lines together, unless there are a lot of description lines. So if there are fewer than 5 of them, we set $rowbreak = '*', which after \FSdesc{} is evaluated, results in the table row delimiter being "\\*". This tells LaTeX to avoid putting a page break immediately after this row. Probably we should set $rowbreak = '' on the last description line to explicitly allow page breaks between packages.

$OUT .= "\\hline\n" if (($line->{'ref'} || 0) ne $lastref);
if ($section->{description_generator}) {
$OUT .= &{$section->{description_generator}}($line);
} elsif ($section->{usage_section}) {
my $minutes = sprintf('%d', $line->{'duration'} / 60);
my $seconds = $line->{'duration'} % 60;
$OUT .= '\FSusagedesc
{' . $line->{'description'} . '}
{' . $line->{'quantity'} . '}
{' . $minutes . 'm ' . $seconds . 's' . '}
{' . $line->{'amount'} . '}';
} else {
$OUT .= '\FSdesc'.
'{}'.
'{' . $line->{'description'} . '}' ;
if ( $unitprices and length($line->{'unit_amount'}) ) {
<nowiki># then show the unit amount and quantity</nowiki>
$OUT .=
'{\\dollar' . $line->{'unit_amount'} . '}'.
'{' . $line->{'quantity'} . '}';
} else {
<nowiki># leave those columns blank</nowiki>
$OUT .= '{}{}';
}
$OUT .= '{\\dollar' . $line->{'amount'} . "}${rowbreak}\n";
}
$lastref = $line->{'ref'} || 0;

Output ext_description lines. We assume that any of them that contain an unescaped "&" are call details and should use \FScalldetail. Otherwise they use \FSextdesc.

foreach my $ext_desc (@$ext_description) {
if ($section->{extended_description_generator}) {
$OUT .= &{$section->{extended_description_generator}}($ext_desc);
} elsif ( $ext_desc !~ /[^\\]&/ ) {
$OUT .= '\FSextdesc{' . $ext_desc . "}$rowbreak\n";
} else { # call detail
$OUT .= '\FScalldetail{' . $ext_desc . "}$rowbreak\n";
}
}
}

End the longtable. If there is a posttotal item in this section (the final Balance Due / Credit Balance message, previous invoice aging, a few other things) then show it right-justified, in the same formatting used for the caption.

$OUT .= '\end{longtable}';
}
if ($section->{'posttotal'}) {
$OUT .= '\begin{flushright}';
$OUT .= '\normalfont\large\bfseries\textsc{'. $section->{'posttotal'}. '}\\\\';
$OUT .= '\\end{flushright}';
}
}

=== Notes ===
For the notes panel, create a minipage the full width of the text. (Unless $summary is in use; then the notes were on the summary page.)

There's some tricky handling of vertical space here. We want the notes at the bottom of the page, regardless of where the last section ended, so the \vfill between them acts as an expandable space.

However, if the notes end up on the first page, we have to prevent them from overprinting the coupon. So if there's a coupon and the notes are on page 1, we insert a strut of height \extracouponspace below the notes. The \vfill will then expand so that the bottom of the strut is at the bottom of the page, keeping the coupon space clear.

--@]
\vfill
\begin{minipage}[t]{\textwidth}
[@-- length($summary)
?
: ( $smallernotes
? '\scriptsize{ '.$notes.' }'
: $notes
)
--@]
[@-- $coupon ? '\ifthenelse{\equal{\thepage}{1}}{\rule{0pt}{\extracouponspace}}{}' : ''--@]''
\end{minipage}
\end{document}

That's the end of the invoice.

== The summary template ==
Start with a two-column table, left side for the notes and right side for the billing summary. On the left side, make a 6.4cm minipage, and put a 10cm vertical strut there, along with the notes.

\begin{tabular}{ll}
\begin{minipage}{6.4cm}
\begin{tabular}{m{0cm}m{6.4cm}}
\rule{0cm}{10cm}&\begin{minipage}{6cm}[@-- $notes --@]\end{minipage}\\
\end{tabular}
\end{minipage} &

On the right side, place a 2cm horizontal strut followed by a 12.8cm minipage. This will contain the billing summary.

\rule{2cm}{0cm}
\begin{minipage}{12.8cm}

The billing summary itself is a two-column table, one for labels and one for amounts. The section headings are just labels with no amounts, with lines underneath.

'true_previous_balance' is, as far as we can determine it, the balance that was printed on the customer's previous invoice. It's the sum of all the customer's transactions (invoices, credits, payments, refunds) dated before that invoice, plus the invoice amount.

'balance_adjustments' is the sum of payments and credits received after the previous invoice, but applied to invoices before this current invoice. 'true_previous_balance' - 'balance_adjustments' is the total amount owed on all past invoices.

\begin{tabular}{lr}
\hline
&\\
\textbf{\underline{Summary of Previous Balance and Payments}} & \\
&\\
\textbf{Previous Balance}&\textbf{\dollar[@-- $true_previous_balance --@]}\\
\textbf{Payments}&\textbf{\dollar[@-- $balance_adjustments --@]}\\
\cline{2-2}
\textbf{Balance Outstanding}&\textbf{\dollar[@-- sprintf('%.2f', $true_previous_balance -$balance_adjustments) --@]}\\
&\\
\hline
&\\
\textbf{\underline{Summary of New Charges}} & \\
&\\

@summary_subtotals contains one line for each invoice section, except for the sections that have special handling such as the adjustments and finance charges. Make a line for each of those subtotals. \cline draws a horizontal line (like \hline) but only under specified columns: here, only the second column. After that, make rows for the total new charges (minus finance charges), the total amount owed on previous invoices ('true_previous_balance' – 'balance_adjustments'), and the finance charges.

[@--
foreach my $section (@summary_subtotals) {
$OUT .= '\textbf{'. ($section->{'description'} ? $section->{'description'} : 'Charges' ). '}';
$OUT .= '&\textbf{'. $section->{'subtotal'}. '}\\\\';
}
$OUT .= '\cline{2-2}';
--@]
\textbf{New Charges Total}&\textbf{\dollar[@-- $current_less_finance --@]}\\
&\\
\hline
&\\
\textbf{\underline{Invoice Summary}} & \\
& \\
\textbf{Previous Past Due Charges}&\textbf{\dollar[@-- sprintf('%.2f', $true_previous_balance - $balance_adjustments) --@]}\\
\textbf{Finance charges on overdue amount}&\textbf{\dollar[@-- $finance_amount --@]}\\
\textbf{New Charges}&\textbf{\dollar[@-- $current_less_finance --@]}\\

Then show the subtotal of the adjustment section, if there is one. This contains payments and credits applied to the ''current'' invoice (not past invoices).

[@--
<nowiki>#false laziness w/invoice_htmlsummary and above</nowiki>
foreach my $section ( grep $_->{adjust_section}, @sections ) {
$OUT .= '\textbf{'. ($section->{'description'} ? $section->{'description'} : 'Charges' ). '}';
$OUT .= '&\textbf{'. $section->{'subtotal'}. '}\\\\';
}
--@]

Finally, the customer's balance including the current invoice, and then close the table and minipage, and force a page break so the line items can start on a new page.

\cline{2-2}
\textbf{Total Amount Due}&\textbf{\dollar[@-- sprintf('%.2f', $balance) --@]}\\
&\\
\hline
\end{tabular}
\end{minipage} \\
\end{tabular}
\newpage

== Invoice modes ==

Invoice template documentation

2017-01-11T03:32:33Z

Mark:

= General notes =
Templates for PDF invoices are in LaTeX, with Perl inclusions in [@-- --@] blocks. Each block can insert a string at its position by either assigning the string to the $OUT variable, or simply returning a string. For details, see Text::Template.

The main function for generating the LaTeX code is FS::Template_Mixin::print_generic. This is a method of the cust_bill object (and also of quotation objects), and takes several arguments described in the perldoc. Most of what this function does is set up the "%invoice_data" hash, which gets imported into local variable space when evaluating the template.

print_generic() actually constructs several templates: 'notes', 'coupon', 'footer', 'smallfooter', 'watermark', and sometimes 'summary'. Exactly which config variable is loaded for each of these is decided in Template_Mixin; look for the line that starts "my $tc". It's '''invoice_$format$part''' (where $format is either "latex" or "html" and $part is "notes", "coupon", etc.). Each template is evaluated, with %invoice_data as arguments, then put back into %invoice_data as "$notes", "$coupon", etc. The main template contains code fragments to include them.

To further complicate things, all of these elements have locale overrides AND invoice mode overrides. If print_generic() was called with a 'mode' argument, those templates will be fetched from the invoice mode + customer locale. If not, they'll be taken from the system config for the customer's agent and locale.

You can obtain the generated LaTeX code for a particular invoice with ''view/cust_bill-tex.cgi'', which takes all the same parameters as ''view/cust_bill.cgi''.

The template mechanics make it difficult to insert graphics other than the logo. During the rendering process, the content of the logo.eps config is written to a file with a random name, and then that name is passed as the 'logo_file' parameter. However, the \includegraphics command has access to the entire filesystem, so one option is to place static graphics somewhere under /home/freeside.

One useful tool for editing a template is Overleaf ([https://www.overleaf.com/ https://www.overleaf.com/]), which is a realtime LaTeX editor. This will require you to upload the logo EPS file with the correct randomly generated name (or edit the references to it) along with the modified longtable.sty file. Also, since it can't evaluate the Perl blocks, you have to work with the generated code rather than the template itself, and then manually edit your changes back into the template. It's still much easier than the cycle of "edit template, set config, download and examine PDF", especially since it will display LaTeX errors in a semi-useful way.

== Positioning ==
This is not used in the standard template but has been used in custom templates. Often in an invoice there's a need to place something at a known position on the page, like a return address that needs to line up with an envelope window. LaTeX isn't designed to facilitate that but LaTeX can do absolutely anything, so: the "textpos" package.

"textpos" requires you to set up a grid system, like this:

\usepackage[absolute]{textpos}
\setlength{\TPHorizModule}{1in}
\setlength{\TPVertModule}{1in}

The environment for a fixed position on the page is "textblock". The syntax is odd; notice parentheses rather than braces, and numbers without units.

\begin{textblock}{3.25}(0.625, 0.5)
\footnotesize{
This text is in a box starting 0.625in left and 0.5in down from the page
origin, in a box 3.25in wide. It will fit inside an envelope window.
}
\end{textblock}

The textblock behaves a lot like a minipage, and can contain almost anything; in particular, \includegraphics works. It's positioned after everything else on the page, which means there's no protection against overprinting anything.

== The standard invoice template ==
=== Preamble ===
Set up the document class and paper size, load all packages we need. In general, all \usepackage commands should be at the start of the document.

* "fancyhdr" supplies the \fancyhead and \fancyfoot commands to place left, right, and center headers and footers.
* "lastpage" creates the LastPage reference for displaying "page X of Y" labels.
* "ifthen" provides the \ifthenelse command.
* "array" provides support for custom table column formats.
* "longtable" provides tables that can span multiple pages. We ship a modified longtable.sty, which reserves vertical space on the first page to allow for the coupon. The amount of vertical space is in the "\LTcouponspace" variable.
* "afterpage" was removed from the template in May 2005 and we can remove the package at some point.
* "multirow" is used to make a table cell in the coupon to hold the return address.
* "bigstrut" appears not to be used.
* "truncate" provides a command to apply a width limit to a line of text, so that anything past that gets replaced with an ellipsis. This is highly encouraged for things like package descriptions that could otherwise overflow their table cells.
* "graphicx" provides the \includegraphics command for inserting the EPS of the logo.
* "inputenc" and "fontenc" make LaTeX play nicely with UTF-8 characters.
* "background" is used to create watermarks (such as are used for some past-due notices, or voided invoices). The \backgroundsetup command takes a TikZ argument list, which is extremely powerful. See "texdoc pgf" for all the horrifying details.

Useful things to add here:

* "textpos": see above.
* "geometry" to manage the margins and headers in a centralized way. It also takes the [showframe] option, which will draw lines to visualize the page layout.
* "xcolor" if you want color.
* "tabularx" and/or "tabulary" add more options for setting table column widths.

\documentclass[letterpaper]{article}

\usepackage{fancyhdr,lastpage,ifthen,array,longtable,afterpage,caption,multirow,bigstrut}
\usepackage[breakwords]{truncate} % to avoid overflowing boxes
\usepackage{graphicx}  % required for logo graphic
\usepackage[utf8]{inputenc}  % multilanguage support
\usepackage[T1]{fontenc}
[@-- if ( length($watermark) ) {
$OUT .= '
\usepackage{background}
\backgroundsetup{
placement=center,
opacity=0.25,
color=black,
angle=0,
contents=' . $watermark . '
}';
}
''<nowiki>;</nowiki>''
--@]

Set up dimensions. Most of these were determined empirically and really ought to use the "geometry" package. A few of the dimensions can be adjusted by config variables: \topmargin, \headsep, \textheight.

See [https://en.wikibooks.org/wiki/LaTeX/Page_Layout LaTeX Page Layout] and [http://mirrors.ctan.org/macros/latex/base/letter.pdf texdoc letter] for how LaTeX thinks a letter-class page is structured. Note that everything is laid out statically from the top of the page. The header starts at (1in + \voffset + \topmargin), and is placed in a box with height \headheight, and then the document body starts at (1in + \voffset + \topmargin + \headheight + \headsep), and then the footer starts (\textheight + \footskip) below that.

If the header doesn't fit within \headheight, LaTeX will complain, and then adjust the headheight, pushing everything else down. There's also not actually any space reserved for the footer; it has to fit into the space left below the body container.

Commands for manipulating length variables: \newlength declares them (and all of these are predeclared in the document stylesheet); \setlength sets them to a value; \addtolength adds to them.

If you're adjusting margins, it can be helpful to add "\usepackage{layout}" to the preamble, and then "\newpage\layout" before the final "\end{document}". This will print a diagram showing the layout boxes and their dimensions.

\LTchunksize is an internal variable in longtable for how many rows to process at a time. There's probably no reason to change it.

\addtolength{\voffset}{-0.0cm}  % top margin to top of header
\addtolength{\hoffset}{-0.6cm}  % left margin on page
\addtolength{\topmargin}{[@-- defined($topmargin) ? $topmargin : '-1.00cm' --@]}
\setlength{\headheight}{2.0cm}  % height of header
\setlength{\headsep}{[@-- defined($headsep) ? $headsep : '1.0cm' --@]}
\setlength{\footskip}{1.0cm}  % bottom of footer from bottom of text

%\addtolength{\textwidth}{2.1in}  % width of text
\setlength{\textwidth}{19.5cm}
\setlength{\textheight}{[@-- defined($textheight) ? $textheight : '19.5cm' --@]}
\setlength{\oddsidemargin}{-0.9cm}  % odd page left margin
\setlength{\evensidemargin}{-0.9cm}  % even page left margin

\LTchunksize=40

$coupon is a local variable containing the entire generated content of the [http://www.freeside.biz/mediawiki/index.php?title=Invoice_coupon_template&action=edit&redlink=1 invoice payment coupon]. If for some reason there isn't a payment coupon (the customer doesn't owe any money, or this is a quotation or statement rather than an invoice) then it's an empty string.

\footrule is the macro used to draw a line at the top of the footer. Here, we redefine it so that it doesn't appear on the first page if there's a payment coupon (because the coupon will make its own line, at a different position).

\extracouponspace is a config setting for how much space to reserve for the coupon. We will "shorten" the first page's \textheight by that much, which moves the starting position of the header up.

\renewcommand{\headrulewidth}{0pt}
\renewcommand{\footrulewidth}{1pt}

\renewcommand{\footrule}{
[@--
$coupon ? '\ifthenelse{\equal{\thepage}{1}}' : ''<nowiki>;</nowiki>''
--@]
{
}
{
\vbox to 0pt{\rule{\headwidth}{\footrulewidth}\vss}
}
}
\newcommand{\extracouponspace}{[@-- defined($extracouponspace) ? $extracouponspace : '2.7in' --@]}

Positioning mailing addresses. This is a bad way to do it and I'd recommend using the "textpos" package instead. Also a command here to output the dollar sign (since '$' has special properties in LaTeX). The escaping functions in Template_Mixin.pm know about this, and will substitute \dollar throughout.

% Adjust the inset of the mailing address
\newcommand{\addressinset}[1][]{\hspace{1.0cm}}

% Adjust the inset of the return address and logo
\newcommand{\returninset}[1][]{\hspace{-0.25cm}}

% New command for address lines i.e. skip them if blank
\newcommand{\addressline}[1]{\ifthenelse{\equal{#1}{}}{}{#1\\}}

% Inserts dollar symbol
\newcommand{\dollar}[1][]{\symbol{36}}

=== Headers/footers ===
"fancyhdr" allows separate left, right, and center headers and footers (the first argument to \fancyhead or \fancyfoot). We use left and right headers and a center footer.

These commands are supposed to remove the default "plain" style page footer, but we don't use that page style so it's probably not needed.

% Remove plain style header/footer
\fancypagestyle{plain}{
\fancyhead{}
}
\fancyhf{}

The center footer. On page 1, if there's a coupon, the coupon is the footer. In that case, we move the start position up by \extracouponspace (using \vspace) so that it starts above the normal footer position.

The footer to display below the coupon (or if there isn't one) is configurable as '''invoice_latexfooter'''. The footer to show on subsequent pages is '''invoice_latexsmallfooter'''. By default they're both just the company name.

% Define fancy header/footer for first and subsequent pages
\fancyfoot[C]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
[@--
if ($coupon) {
$OUT .= '\vspace{-\extracouponspace}';
$OUT .= '\rule[0.5em]{\textwidth}{\footrulewidth}\\\\';
$OUT .= $coupon;
$OUT .= '\vspace{'.
(defined($couponfootsep) ? $couponfootsep : '0.2in') .
'}';
}
''<nowiki>;</nowiki>''
--@] [@-- $smallerfooter ? '\scriptsize{' : '\small{' --@]
[@-- $footer --@]
}[@-- $coupon ? '\vspace{\extracouponspace}' : ''--@]''
}
{ % ... pages
[@-- $smallerfooter ? '\scriptsize{' : '\small{' --@]
[@-- $smallfooter --@]
}
}
}

The right side footer, showing the page number. "emt()" localizes text to the customer's language and escapes for LaTeX. "~" is a non-breaking space character.

\fancyfoot[R]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
}
{ % ... pages
\small{\thepage~[@-- emt('of') --@]~\pageref{LastPage}}
}
}

The left side header, containing the return address and logo on the first page (and nothing on later pages). \returninset is just a horizontal adjustment. \makebox{} tells LaTeX to put its contents into a single box. That's probably unnecessary because the only thing in the box is a table, which is already a single box.

\begin{tabular}{ll} creates a two-column table. In the first column is a "minipage", which is just a fixed-width container environment (5.5cm here) with its bottom edge aligned to the table cell it's in. In here we put the evaluated "returnaddress" template.

Tables in LaTeX always use & to separate columns and \\ to separate rows. A row must always contain as many columns as in the column spec; otherwise LaTeX will guess about what you mean and will usually get it wrong. \\ at the end of the last row is optional.

The second column contains the logo graphic. See "texdoc graphicx" for a full explanation of this command; it can do many useful things including scaling and repositioning the graphic. For example, if you want the logo to be scaled to be 0.5 inch tall, you can do this: "\includegraphics[height=0.5in]{[@-- $logo_file --@]}".

\fancyhead[L]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
\returninset
\makebox{
\begin{tabular}{ll}
\begin{minipage}[b]{5.5cm}
[@-- $returnaddress --@]
\end{minipage} &
\includegraphics{[@-- $logo_file --@]}\\
\end{tabular}
}
}
{ % ... pages
 %\includegraphics{[@-- $logo_file --@]} % Uncomment if you want the logo on all pages.
}
}

The right side header, containing the invoice date, invnum, and custnum in a three-column table with each column centered ({ccc}). "$no_date" and "$no_number" are both enabled by arguments to print_generic() and used when printing statements.

Note that "$date" is already formatted using time2str_local('long'), so something like "Feb 1st, 2017". The template can call time2str() to format other dates; this is an alias to time2str_local so it will use the customer's language settings and LaTeX-escape the result.

This is followed by a horizontal line spanning the table (\hline), then a row where the middle cell contains the $notice_name (usually "Invoice") in \huge font, in small caps (\textsc). The \rule{0pt}{5ex} creates a vertical strut (a zero-width rule) which increases the row height to 5ex. Then another \hline.

Note that \hline should never be followed by \\ and will cause errors if it is.

On subsequent pages we show a smaller version without the notice name.

\fancyhead[R]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
\begin{tabular}{ccc}
[@-- join(' & ', ( $no_date  ?  '': emt('Invoice date') ),''
( $no_number ?  '': emt('Invoice #') ),''
emt('Customer #')
)
--@]\\
\vspace{0.2cm}
\textbf{[@-- $date --@]} & \textbf{[@-- $invnum --@]} & \textbf{[@-- $custnum --@]} \\\hline
\rule{0pt}{5ex} &~~ \huge{\textsc{[@-- emt($notice_name) --@]}} & \\
\vspace{-0.2cm}
& & \\\hline
\end{tabular}
}
{ % ... pages
\small{
\begin{tabular}{lll}
[@-- join(' & ', emt('Invoice date'), emt('Invoice #'), emt('Customer #') ) --@]\\
\textbf{[@-- $date --@]} & \textbf{[@-- $invnum --@]} & \textbf{[@-- $custnum --@]}\\
\end{tabular}
}
}
}

Enable the fancyhdr page style and set the font family.

\pagestyle{fancy}
%% Font options are:
%% bch Bitsream Charter
%% put Utopia
%% phv Adobe Helvetica
%% pnc New Century Schoolbook
%% ptm Times
%% pcr Courier

\renewcommand{\familydefault}{phv}

=== Line item table macros ===
This section is Freeside-specific. The main invoice line item table is an 8-column longtable. Normally the columns are:

* Package number
* Description (6 columns)
* Price

(The 6 columns are used separately for showing CDRs, if there are any.)

If '''invoice-unitprice''' is on, it's:

* Package number
* Description (4 columns)
* Unit price
* Quantity
* Price

According to that setting, create macros for how many columns and how much space to allocate to the description, and for the headers of the Unit Price and Unit Quantity columns.

% Commands for freeside table header...

\newcommand{\FSdescriptionlength} { [@-- $unitprices ? '8.2cm' : '12.8cm' --@] }
\newcommand{\FSdescriptioncolumncount} { [@-- $unitprices ? '4' : '6' --@] }
\newcommand{\FSunitcolumns}{ [@--
$unitprices
 ? '\makebox[2.5cm][r]{\textbf{~~' . emt('Unit Price') . '}} &' .
'\makebox[1.4cm]{\textbf{~' . emt('Quantity') . '}} & '
 : ''--@] }''

\FShead renders the main table header. Using \makebox inside a column forces it to a minimum width. \multicolumn{N}{X}{ ... } is like COLSPAN in HTML: makes a cell spanning N columns. The second argument is how to align the contents of the column (l, r, or c).

\truncate renders its argument into a space with a maximum width. It will break at word boundaries and insert an ellipsis afterward. Note that if we didn't do this, longtable would either squish the surrounding columns or push them off the page rather than word-wrap the description.

\FSusagehead is the same, but used for usage summary sections that show the number of calls and total minutes. The '''usage_class_summary''' config turns this on.

\newcommand{\FShead}{
\hline
\rule{0pt}{2.5ex}
\makebox[1.4cm]{} &
\multicolumn{\FSdescriptioncolumncount}{l}{
\truncate{\FSdescriptionlength}{\textbf{[@-- emt('Description') --@]}}
} &
\FSunitcolumns
\makebox[1.6cm][r]{\textbf{[@-- emt('Amount') --@]}} \\
\hline
}

\newcommand{\FSusagehead}{
\hline
\rule{0pt}{2.5ex}
\makebox[1.4cm]{} &
\multicolumn{4}{l}{
\truncate{\FSdescriptionlength}{\textbf{[@-- emt('Description') --@]}}
} &
\textbf{~~[@-- emt('Calls') --@]} &
\textbf{~~[@-- emt('Duration') --@]} &
\textbf{~~[@-- emt('Amount') --@]} \\
\hline
}

Next, the commands to actually display line items.

\FSdesc will be called once for each element in the @detail_items array. This takes five arguments (that's the [5] in the declaration) which are the values to put in the pkgnum, description, unit price, quantity, and price columns. If unit prices are off then arguments 3 and 4 will be empty.

\FSextdesc will be called once for each element in the 'ext_description' element of the detail item. These contain things like service labels, discount details, prorate details, and manually created package details. This macro takes a single argument and places it in a big multicolumn below the description, in small font.

\FScalldetail is used only for call details. These are preprocessed by FS::TemplateItem_Mixin::details() ''into LaTeX table rows'' (pre-escaped, delimited with &). So, the macro just skips the first column and outputs its argument.

\FStotaldesc is used for subtotal and total rows; it puts its first argument in a big multicolumn in the description space, and the second in the amount column.

Finally, \FSusagedesc is for usage summary sections, and puts the total calls, duration, and amount in the correct columns.

% ...description...
\newcommand{\FSdesc}[5]{
\multicolumn{1}{c}{\rule{0pt}{2.5ex}\textbf{#1}} &
\multicolumn{[@-- $unitprices ? '4' : '6' --@]}{l}{
\truncate{\FSdescriptionlength}{\textbf{#2}}
} &
[@-- $unitprices ? ' \multicolumn{1}{r}{\textbf{#3}} &'."\n".
' \multicolumn{1}{r}{\textbf{#4}} &'."\n"
 :
--@]
\multicolumn{1}{r}{\textbf{#5}}\\
}
% ...extended description...
\newcommand{\FSextdesc}[1]{
\multicolumn{1}{l}{\rule{0pt}{1.0ex}} &
\multicolumn{6}{l}{
\truncate{12.8cm}{\small{[http://www.freeside.biz/mediawiki/index.php?title=User:Mark&action=edit&redlink=1 Mark] ([http://www.freeside.biz/mediawiki/index.php?title=User_talk:Mark&action=edit&redlink=1 talk])#1}}
} \\
}
% ...call detail (multiple columns already)...
\newcommand{\FScalldetail}[1]{
\multicolumn{1}{l}{\rule{0pt}{1.0ex}} &
[http://www.freeside.biz/mediawiki/index.php?title=User:Mark&action=edit&redlink=1 Mark] ([http://www.freeside.biz/mediawiki/index.php?title=User_talk:Mark&action=edit&redlink=1 talk])#1
\\
}
}
% ...and total line items (which use the full 12.8cm length, ignoring
% unitprice/quantity
\newcommand{\FStotaldesc}[2]{
& \multicolumn{6}{l}{
\truncate{12.8cm}{#1}
} & #2\\
}
% ...usage class summary
\newcommand{\FSusagedesc}[4]{
\multicolumn{1}{c}{\rule{0pt}{2.5ex}} &
\multicolumn{4}{l}{\textbf{#1}} &
\multicolumn{1}{r}{\textbf{#2}} &
\multicolumn{1}{r}{\textbf{#3}} &
\multicolumn{1}{r}{\textbf{#4}}
\\
}

=== Main document ===
First, make a minipage for the customer's name and address. This starts at \addressinset + 5 mm (not sure why) and can be up to 7 cm wide. The \makebox does nothing (as above, a minipage is already a single box). The \addressline macro makes a line break only if the argument is non-empty, to avoid errors for breaking an empty line.

\begin{document}
% Headers and footers defined for the first page
\addressinset \rule{0.5cm}{0cm}
\makebox{
\begin{minipage}[t]{7.0cm}
\vspace{[@-- defined($addresssep) ? $addresssep : '0.25cm' --@]}
\textbf{[@-- $payname --@]}\\
\addressline{[@-- $company --@]}
\addressline{[@-- $address1 --@]}
\addressline{[@-- $address2 --@]}
\addressline{[@-- $city --@], [@-- $state --@]~~[@-- $zip --@]}
\addressline{[@-- $country --@]}
\end{minipage}}

Make another minipage for the service address, terms, and PO number. $ship_enable is true if the global '''invoice-ship_address''' or per-customer "invoice_ship_address" flag is on.

The initial \hfill spreads out the two minipages so that they go all the way to the margins. The 'terms' and 'po_line' should probably be in \addressline{} macros but currently aren't.

After that, skip some vertical space. (If you need more vertical space for an invoice format, the 1.5cm here can be cut.)

\hfill
\makebox{
\begin{minipage}[t]{6.4cm}
[@--
if ($ship_enable) {
$OUT .= '\textbf{' . emt('Service Address') . '}\\\\';
$OUT .= "\\addressline{$ship_company}";
$OUT .= "\\addressline{$ship_address1}";
$OUT .= "\\addressline{$ship_address2}";
$OUT .= "\\addressline{$ship_city, $ship_state~~$ship_zip}";
$OUT .= "\\addressline{$ship_country}";
$OUT .= '~\\\\';
}else{
$OUT .= ''<nowiki>;</nowiki>''
}
--@]
\begin{flushright}
[@-- $terms ? emt('Terms') . ': ' . emt($terms) : ''--@]\\''
[@-- $po_line --@]\\
\end{flushright}
\end{minipage}}
\vspace{1.5cm}

Insert the [http://www.freeside.biz/mediawiki/index.php?title=Summary&action=edit&redlink=1 Summary] if there is one. The last thing in the summary template is a page break.

%
[@-- $summary --@]
%

=== Sections ===
Start creating sections, according to the @sections array. Section entries contain the following keys:

* description
* category (pkg_category.categoryname, for package sections by category)
* location (hashref of location fields, for package sections by location)
* pretotal (formatted with \FStotaldesc, before detail lines)
* subtotal (formatted with \FStotaldesc, at the end)
* posttotal (line to display after the end of the section; used for "Balance Due" message and a few other things)
* sort_weight (sort order, but also does complicated things)

and optionally some flags:

* post_total (put the section at the end of the invoice)
* summarized (omit the section entirely)
* usage_section (use \FSusagedesc instead of \FSdesc, see above)
* no_subtotal (omit the subtotal line)

If '''invoice_sections''' is enabled: the $multisection flag in print_generic() is turned on, and @sections is filled by calling FS::Template_Mixin::_items_sections, which makes a section for each package category. Line items for packages go into the package category's section. '''invoice_sections_method''' can be used to divide sections by location rather than by category. In addition, sections are created for previous invoices, taxes, and "adjustments" (credits and payments to this invoice).

If not: a single section is created, with empty description, containing previous invoices, current charges, and adjustments (after the total). '''previous_balance-section''' will make a previous invoices section anyway.

Additional sections are created in some cases. '''svc_phone_sections''' creates a section for each phone line (see RT#6592; note that this customer is no longer active). '''voip-cust_accountcode_cdr''' creates a section for each accountcode (see RT#12181). '''discount-show_available''' creates a section with information on available discounts. '''usage_class_summary''' creates a section with usage subtotals.

'''finance_pkgclass''' specifies the package class containing "finance charges", which is handled specially. That class's category name is passed to the template as $finance_section, and the sum of charges in that category is passed as $finance_amount. That amount is broken out from the "Current charges" line in the summary section. On a multisection summary-format invoice, the finance charges will also ''not'' be shown in the body of the invoice. I don't know why.

Loop over all sections (except for the finance charges, maybe). Inside the loop we'll do everything by appending to $OUT.

\section*{}
[@--
foreach my $section ( grep { !$summary || $_->{description} ne $finance_section } @sections ) {

We're going to do many things if "!$summary". If there's a summary section, the pretotals and posttotals are redundant so skip them.

if ($section->{'pretotal'} && !$summary) {
$OUT .= '\begin{flushright}';
$OUT .= '\large\textsc{'. $section->{'pretotal'}. '}\\\\';
$OUT .= '\\end{flushright}';
}

Start a new page if this is a "post_total" section. As far as I know these are used only for usage details. I think the 'summarized' flag is no longer used for anything.

The section heading is going to be a caption placed inside the main table. \captionsetup sets properties for the caption (left justified, bold, small caps, Large size). If this is the first page, also tell longtable (via \LTextracouponspace) to shorten the effective length of the page so that it doesn't overprint the coupon.

$OUT .= '\pagebreak' if $section->{'post_total'};
unless ($section->{'summarized'} ) {
$OUT .= '\captionsetup{singlelinecheck=false,justification=raggedright,font={Large,sc,bf}}';
$OUT .= '\ifthenelse{\equal{\thepage}{1}}{\setlength{\LTextracouponspace}{\extracouponspace}}{\setlength{\LTextracouponspace}{0pt}}'
if $coupon;

Start the longtable that will contain the section's line items. Eight columns, pkgnum (centered), description and optional unitprice/quantity (left), then price (right).

\caption* produces a table caption without prefixing it with "Table 1". The caption is the section description, or 'Charges' if it's null (except that if packages are sectioned by location, we put one together from the location fields).

$OUT .= '\begin{longtable}{cllllllr}';
$OUT .= '\caption*{ ';
if ($section->{'location'}) {
$OUT .= $section->{'location'}{'label_prefix'}. ': '
if length($section->{'location'}{'label_prefix'});
$OUT .= $section->{'location'}{'address1'};
$OUT .= ', ' . $section->{'location'}{'address2'}
if length($section->{'location'}{'address2'});
$OUT .= ', ' .
$section->{'location'}{'city'} . ', ' .
$section->{'location'}{'state'} . '~' .
$section->{'location'}{'zip'};
} elsif ( $section->{'description'} ) {
$OUT .= ($section->{'description'});
} else {
$OUT .= emt('Charges');
}
$OUT .= '}\\\\';

The point of a longtable is that it can span multiple pages, including printing head/foot rows at the top/bottom of each page automatically. The protocol for setting these up is to specify the first head, per-page head, per-page foot, and last foot, then the rest of the table contents. Here, we set the first head to be \FShead (or \FSusagehead). \endfirsthead means we're done specifying it.

"header_generator" and other foo_generators are from an old mechanism that injected callbacks into the template data. This is strongly deprecated.

if ($section->{header_generator}) {
$OUT .= &{$section->{header_generator}}();
} elsif ( $section->{usage_section} ) {
$OUT .= '\FSusagehead';
} else {
$OUT .= '\FShead';
}
$OUT .= '\endfirsthead';

The per-page head: a row with a "Continued" message, then \FShead again. The per-page foot is the same.

$OUT .= '\multicolumn{7}{r}{\rule{0pt}{2.5ex}'.emt('Continued from previous page').'}\\\\';
if ($section->{header_generator}) {
$OUT .= &{$section->{header_generator}}();
} elsif ( $section->{usage_section} ) {
$OUT .= '\FSusagehead';
} else {
$OUT .= '\FShead';
}
$OUT .= '\endhead';
$OUT .= '\multicolumn{7}{r}{\rule{0pt}{2.5ex}'.emt('Continued on next page...').'}\\\\';
$OUT .= '\endfoot';
$OUT .= '\hline';

The final foot for the section. For multisection invoices this contains the section subtotal:

if (scalar(@sections) > 1 and !$section->{no_subtotal}) {
if ($section->{total_generator}) {
$OUT .= &{$section->{total_generator}}($section);
} else {
$OUT .= '\FStotaldesc{' . $section->{'description'} . ' Total}' .
'{' . $section->{'subtotal'} . '}' . "\n";
}
}

Single-section invoices use "@total_items" for items that need to appear after the line items. These include the pre-tax subtotal, taxes, total charges, adjustments, and the balance due. Call \FStotaldesc for each of them. Then end the section foot.

<nowiki>#if ($section == $sections[$#sections]) {</nowiki>
foreach my $line (grep {$_->{section}->{description} eq $section->{description}} @total_items) {
if ($section->{total_line_generator}) {
$OUT .= &{$section->{total_line_generator}}($line);
} else {
$OUT .= '\FStotaldesc{' . $line->{'total_item'} . '}' .
'{' . $line->{'total_amount'} . '}' . "\n";
}
}
<nowiki>#}</nowiki>
$OUT .= '\hline';
$OUT .= '\endlastfoot';

Line items are passed in @detail_items. A line item entry can contain:

* section (reference)
* ref (pkgnum)
* description
* ext_description (arrayref of additional lines)
* quantity
* amount
* duration (for usage summary sections)
* pkgpart
* unit_amount (unit price)
* locationnum
* svc_label (the primary service's label)

Line items are created in several places in Template_Mixin and TemplateItem_Mixin, most importantly _items_cust_bill_pkg.

@detail_items is a single array; we find the lines belonging to this section by checking their 'section' element. If there's only one section, process all the lines.

my $lastref = 0;
foreach my $line (
grep { ( scalar( @sections ) > 1
 ? $section->{'description'} eq $_->{'section'}->{'description'}
 : 1
) }
@detail_items )
{
my $ext_description = $line->{'ext_description'};

<nowiki># Don't break-up small packages.</nowiki>
my $rowbreak = @$ext_description < 5 ? '*' : ''<nowiki>;</nowiki>''

$lastref is set to the pkgnum of the previous line item. If the value of 'ref' no longer equals that, we're now showing a different package; separate with an \hline. After outputting this line item we'll set $lastref to the new pkgnum.

If this is a usage summary section, convert the duration to minutes/seconds and call \FSusagedesc, passing the description, quantity (number of calls), duration, and total price.

If it's a normal section, call \FSdesc, passing the description, unit price, quantity, and amount. If unit prices aren't in use, or there isn't one on this line item, then leave unit price and quantity blank.

"$rowbreak" is a flag for whether to allow a page break. In general we want to keep line items and their ext_description lines together, unless there are a lot of description lines. So if there are fewer than 5 of them, we set $rowbreak = '*', which after \FSdesc{} is evaluated, results in the table row delimiter being "\\*". This tells LaTeX to avoid putting a page break immediately after this row. Probably we should set $rowbreak = ''on the last description line to explicitly allow page breaks between packages.''

$OUT .= "\\hline\n" if (($line->{'ref'} || 0) ne $lastref);
if ($section->{description_generator}) {
$OUT .= &{$section->{description_generator}}($line);
} elsif ($section->{usage_section}) {
my $minutes = sprintf('%d', $line->{'duration'} / 60);
my $seconds = $line->{'duration'} % 60;
$OUT .= '\FSusagedesc
{' . $line->{'description'} . '}
{' . $line->{'quantity'} . '}
{' . $minutes . 'm ' . $seconds . 's' . '}
{' . $line->{'amount'} . '}';
} else {
$OUT .= '\FSdesc'.
'{}'.
'{' . $line->{'description'} . '}' ;
if ( $unitprices and length($line->{'unit_amount'}) ) {
<nowiki># then show the unit amount and quantity</nowiki>
$OUT .=
'{\\dollar' . $line->{'unit_amount'} . '}'.
'{' . $line->{'quantity'} . '}';
} else {
<nowiki># leave those columns blank</nowiki>
$OUT .= '{}{}';
}
$OUT .= '{\\dollar' . $line->{'amount'} . "}${rowbreak}\n";
}
$lastref = $line->{'ref'} || 0;

Output ext_description lines. We assume that any of them that contain an unescaped "&" are call details and should use \FScalldetail. Otherwise they use \FSextdesc.

foreach my $ext_desc (@$ext_description) {
if ($section->{extended_description_generator}) {
$OUT .= &{$section->{extended_description_generator}}($ext_desc);
} elsif ( $ext_desc !~ /[^\\]&/ ) {
$OUT .= '\FSextdesc{' . $ext_desc . "}$rowbreak\n";
} else { # call detail
$OUT .= '\FScalldetail{' . $ext_desc . "}$rowbreak\n";
}
}
}

End the longtable. If there is a posttotal item in this section (the final Balance Due / Credit Balance message, previous invoice aging, a few other things) then show it right-justified, in the same formatting used for the caption.

$OUT .= '\end{longtable}';
}
if ($section->{'posttotal'}) {
$OUT .= '\begin{flushright}';
$OUT .= '\normalfont\large\bfseries\textsc{'. $section->{'posttotal'}. '}\\\\';
$OUT .= '\\end{flushright}';
}
}

=== Notes ===
For the notes panel, create a minipage the full width of the text. (Unless $summary is in use; then the notes were on the summary page.)

There's some tricky handling of vertical space here. We want the notes at the bottom of the page, regardless of where the last section ended, so the \vfill between them acts as an expandable space.

However, if the notes end up on the first page, we have to prevent them from overprinting the coupon. So if there's a coupon and the notes are on page 1, we insert a strut of height \extracouponspace below the notes. The \vfill will then expand so that the bottom of the ''strut'' is at the bottom of the page, keeping the coupon space clear.

--@]
\vfill
\begin{minipage}[t]{\textwidth}
[@-- length($summary)
 ?
 : ( $smallernotes
 ? '\scriptsize{ '.$notes.' }'
 : $notes
)
--@]
[@-- $coupon ? '\ifthenelse{\equal{\thepage}{1}}{\rule{0pt}{\extracouponspace}}{}' : ''--@]''
\end{minipage}
\end{document}

That's the end of the invoice.

== The summary template ==
Start with a two-column table, left side for the notes and right side for the billing summary. On the left side, make a 6.4cm minipage, and put a 10cm vertical strut there, along with the notes.

\begin{tabular}{ll}
\begin{minipage}{6.4cm}
\begin{tabular}{m{0cm}m{6.4cm}}
\rule{0cm}{10cm}&\begin{minipage}{6cm}[@-- $notes --@]\end{minipage}\\
\end{tabular}
\end{minipage} &

On the right side, place a 2cm horizontal strut followed by a 12.8cm minipage. This will contain the billing summary.

\rule{2cm}{0cm}
\begin{minipage}{12.8cm}

The billing summary itself is a two-column table, one for labels and one for amounts. The section headings are just labels with no amounts, with lines underneath.

'true_previous_balance' is, as far as we can determine it, the balance that was printed on the customer's previous invoice. It's the sum of all the customer's transactions (invoices, credits, payments, refunds) dated before that invoice, plus the invoice amount.

'balance_adjustments' is the sum of payments and credits received after the previous invoice, but applied to invoices before this current invoice. 'true_previous_balance' - 'balance_adjustments' is the total amount owed on all past invoices.

\begin{tabular}{lr}
\hline
&\\
\textbf{\underline{Summary of Previous Balance and Payments}} & \\
&\\
\textbf{Previous Balance}&\textbf{\dollar[@-- $true_previous_balance --@]}\\
\textbf{Payments}&\textbf{\dollar[@-- $balance_adjustments --@]}\\
\cline{2-2}
\textbf{Balance Outstanding}&\textbf{\dollar[@-- sprintf('%.2f', $true_previous_balance -$balance_adjustments) --@]}\\
&\\
\hline
&\\
\textbf{\underline{Summary of New Charges}} & \\
&\\

@summary_subtotals contains one line for each invoice section, except for the sections that have special handling such as the adjustments and finance charges. Make a line for each of those subtotals. This is followed with \cline{2-2}, which puts a horizontal line under the second column only, and then a row for the total of new charges.

[@--
foreach my $section (@summary_subtotals) {
$OUT .= '\textbf{'. ($section->{'description'} ? $section->{'description'} : 'Charges' ). '}';
$OUT .= '&\textbf{'. $section->{'subtotal'}. '}\\\\';
}
$OUT .= '\cline{2-2}';
--@]
\textbf{New Charges Total}&\textbf{\dollar[@-- $current_less_finance --@]}\\
&\\
\hline
&\\
\textbf{\underline{Invoice Summary}} & \\
& \\
\textbf{Previous Past Due Charges}&\textbf{\dollar[@-- sprintf('%.2f', $true_previous_balance - $balance_adjustments) --@]}\\
\textbf{Finance charges on overdue amount}&\textbf{\dollar[@-- $finance_amount --@]}\\
\textbf{New Charges}&\textbf{\dollar[@-- $current_less_finance --@]}\\

Then show the subtotal of the adjustment section, if there is one. This contains payments and credits applied to the ''current'' invoice.

[@--
<nowiki>#false laziness w/invoice_htmlsummary and above</nowiki>
foreach my $section ( grep $_->{adjust_section}, @sections ) {
$OUT .= '\textbf{'. ($section->{'description'} ? $section->{'description'} : 'Charges' ). '}';
$OUT .= '&\textbf{'. $section->{'subtotal'}. '}\\\\';
}
--@]

Finally, the customer's balance including the current invoice, and then close the table and minipage, and force a page break so the line items can start on a new page.

\cline{2-2}
\textbf{Total Amount Due}&\textbf{\dollar[@-- sprintf('%.2f', $balance) --@]}\\
&\\
\hline
\end{tabular}
\end{minipage} \\
\end{tabular}
\newpage

== Invoice modes ==
* [http://www.freeside.biz/mediawiki/index.php?title=Special:UserLogin&returnto=Invoice+template+documentation Log in]

* [http://www.freeside.biz/mediawiki/index.php/Invoice_template_documentation Page]
* [http://www.freeside.biz/mediawiki/index.php?title=Talk:Invoice_template_documentation&action=edit&redlink=1 Discussion]

* [http://www.freeside.biz/mediawiki/index.php/Invoice_template_documentation Read]
* [http://www.freeside.biz/mediawiki/index.php?title=Invoice_template_documentation&action=edit View source]
* [http://www.freeside.biz/mediawiki/index.php?title=Invoice_template_documentation&action=history View history]

===== Navigation =====
* [http://www.freeside.biz/mediawiki/index.php/Main_Page Main page]
* [http://www.freeside.biz/mediawiki/index.php/Special:RecentChanges Recent changes]
* [http://www.freeside.biz/mediawiki/index.php/Special:Random Random page]

===== Tools =====
* [http://www.freeside.biz/mediawiki/index.php/Special:WhatLinksHere/Invoice_template_documentation What links here]
* [http://www.freeside.biz/mediawiki/index.php/Special:RecentChangesLinked/Invoice_template_documentation Related changes]
* [http://www.freeside.biz/mediawiki/index.php/Special:SpecialPages Special pages]
* [http://www.freeside.biz/mediawiki/index.php?title=Invoice_template_documentation&printable=yes Printable version]
* [http://www.freeside.biz/mediawiki/index.php?title=Invoice_template_documentation&oldid=9615 Permanent link]

* This page was last modified on 10 January 2017, at 14:06.
* This page has been accessed 5 times.

* [http://www.freeside.biz/mediawiki/index.php/Freeside:Privacy_policy Privacy policy]
* [http://www.freeside.biz/mediawiki/index.php/Freeside:About About Freeside]
* [http://www.freeside.biz/mediawiki/index.php/Freeside:General_disclaimer Disclaimers]

* [[Image:]]

Invoice template documentation

2017-01-10T22:06:46Z

Mark:

= General notes =

Templates for PDF invoices are in LaTeX, with Perl inclusions in [@-- --@] blocks. Each block can insert a string at its position by either assigning the string to the $OUT variable, or simply returning a string. For details, see Text::Template.

The main function for generating the LaTeX code is FS::Template_Mixin::print_generic. This is a method of the cust_bill object (and also of quotation objects), and takes several arguments described in the perldoc. Most of what this function does is set up the "%invoice_data" hash, which gets imported into local variable space when evaluating the template.

print_generic() actually constructs several templates: 'notes', 'coupon', 'footer', 'smallfooter', 'watermark', and sometimes 'summary'. Exactly which config variable is loaded for each of these is decided in Template_Mixin; look for the line that starts "my $tc". It's '''invoice_$format$part''' (where $format is either "latex" or "html" and $part is "notes", "coupon", etc.). Each template is evaluated, with %invoice_data as arguments, then put back into %invoice_data as "$notes", "$coupon", etc. The main template contains code fragments to include them.

To further complicate things, all of these elements have locale overrides AND invoice mode overrides. If print_generic() was called with a 'mode' argument, those templates will be fetched from the invoice mode + customer locale. If not, they'll be taken from the system config for the customer's agent and locale.

You can obtain the generated LaTeX code for a particular invoice with ''view/cust_bill-tex.cgi'', which takes all the same parameters as ''view/cust_bill.cgi''.

The template mechanics make it difficult to insert graphics other than the logo. During the rendering process, the content of the logo.eps config is written to a file with a random name, and then that name is passed as the 'logo_file' parameter. However, the \includegraphics command has access to the entire filesystem, so one option is to place static graphics somewhere under /home/freeside.

One useful tool for editing a template is Overleaf (https://www.overleaf.com/), which is a realtime LaTeX editor. This will require you to upload the logo EPS file with the correct randomly generated name (or edit the references to it) along with the modified longtable.sty file. Also, since it can't evaluate the Perl blocks, you have to work with the generated code rather than the template itself, and then manually edit your changes back into the template. It's still much easier than the cycle of "edit template, set config, download and examine PDF", especially since it will display LaTeX errors in a semi-useful way.

== Positioning ==

This is not used in the standard template but has been used in custom templates. Often in an invoice there's a need to place something at a known position on the page, like a return address that needs to line up with an envelope window. LaTeX isn't designed to facilitate that but LaTeX can do absolutely anything, so: the "textpos" package.

"textpos" requires you to set up a grid system, like this:

\usepackage[absolute]{textpos}
\setlength{\TPHorizModule}{1in}
\setlength{\TPVertModule}{1in}

The environment for a fixed position on the page is "textblock". The syntax is odd; notice parentheses rather than braces, and numbers without units.

\begin{textblock}{3.25}(0.625, 0.5)
\footnotesize{
This text is in a box starting 0.625in left and 0.5in down from the page
origin, in a box 3.25in wide. It will fit inside an envelope window.
}
\end{textblock}

The textblock behaves a lot like a minipage, and can contain almost anything; in particular, \includegraphics works. It's positioned after everything else on the page, which means there's no protection against overprinting anything.

== The standard invoice template ==

=== Preamble ===

Set up the document class and paper size, load all packages we need. In general, all \usepackage commands should be at the start of the document.

* "fancyhdr" supplies the \fancyhead and \fancyfoot commands to place left, right, and center headers and footers.
* "lastpage" creates the LastPage reference for displaying "page X of Y" labels.
* "ifthen" provides the \ifthenelse command.
* "array" provides support for custom table column formats.
* "longtable" provides tables that can span multiple pages. We ship a modified longtable.sty, which reserves vertical space on the first page to allow for the coupon. The amount of vertical space is in the "\LTcouponspace" variable.
* "afterpage" was removed from the template in May 2005 and we can remove the package at some point.
* "multirow" is used to make a table cell in the coupon to hold the return address.
* "bigstrut" appears not to be used.
* "truncate" provides a command to apply a width limit to a line of text, so that anything past that gets replaced with an ellipsis. This is highly encouraged for things like package descriptions that could otherwise overflow their table cells.
* "graphicx" provides the \includegraphics command for inserting the EPS of the logo.
* "inputenc" and "fontenc" make LaTeX play nicely with UTF-8 characters.
* "background" is used to create watermarks (such as are used for some past-due notices, or voided invoices). The \backgroundsetup command takes a TikZ argument list, which is extremely powerful. See "texdoc pgf" for all the horrifying details.

Useful things to add here:

* "textpos": see above.
* "geometry" to manage the margins and headers in a centralized way. It also takes the [showframe] option, which will draw lines to visualize the page layout.
* "xcolor" if you want color.
* "tabularx" and/or "tabulary" add more options for setting table column widths.

\documentclass[letterpaper]{article}

\usepackage{fancyhdr,lastpage,ifthen,array,longtable,afterpage,caption,multirow,bigstrut}
\usepackage[breakwords]{truncate} % to avoid overflowing boxes
\usepackage{graphicx} % required for logo graphic
\usepackage[utf8]{inputenc} % multilanguage support
\usepackage[T1]{fontenc}
[@-- if ( length($watermark) ) {
$OUT .= '
\usepackage{background}
\backgroundsetup{
placement=center,
opacity=0.25,
color=black,
angle=0,
contents=' . $watermark . '
}';
}
'';
--@]

Set up dimensions. Most of these were determined empirically and really ought to use the "geometry" package. A few of the dimensions can be adjusted by config variables: \topmargin, \headsep, \textheight.

See [https://en.wikibooks.org/wiki/LaTeX/Page_Layout LaTeX Page Layout] and [http://mirrors.ctan.org/macros/latex/base/letter.pdf texdoc letter] for how LaTeX thinks a letter-class page is structured. Note that everything is laid out statically from the top of the page. The header starts at (1in + \voffset + \topmargin), and is placed in a box with height \headheight, and then the document body starts at (1in + \voffset + \topmargin + \headheight + \headsep), and then the footer starts (\textheight + \footskip) below that.

If the header doesn't fit within \headheight, LaTeX will complain, and then adjust the headheight, pushing everything else down. There's also not actually any space reserved for the footer; it has to fit into the space left below the body container.

Commands for manipulating length variables: \newlength declares them (and all of these are predeclared in the document stylesheet); \setlength sets them to a value; \addtolength adds to them.

If you're adjusting margins, it can be helpful to add "\usepackage{layout}" to the preamble, and then "\newpage\layout" before the final "\end{document}". This will print a diagram showing the layout boxes and their dimensions.

\LTchunksize is an internal variable in longtable for how many rows to process at a time. There's probably no reason to change it.

\addtolength{\voffset}{-0.0cm} % top margin to top of header
\addtolength{\hoffset}{-0.6cm} % left margin on page
\addtolength{\topmargin}{[@-- defined($topmargin) ? $topmargin : '-1.00cm' --@]}
\setlength{\headheight}{2.0cm} % height of header
\setlength{\headsep}{[@-- defined($headsep) ? $headsep : '1.0cm' --@]}
\setlength{\footskip}{1.0cm} % bottom of footer from bottom of text

%\addtolength{\textwidth}{2.1in} % width of text
\setlength{\textwidth}{19.5cm}
\setlength{\textheight}{[@-- defined($textheight) ? $textheight : '19.5cm' --@]}
\setlength{\oddsidemargin}{-0.9cm} % odd page left margin
\setlength{\evensidemargin}{-0.9cm} % even page left margin

\LTchunksize=40

$coupon is a local variable containing the entire generated content of the [[Invoice_coupon_template|invoice payment coupon]]. If for some reason there isn't a payment coupon (the customer doesn't owe any money, or this is a quotation or statement rather than an invoice) then it's an empty string.

\footrule is the macro used to draw a line at the top of the footer. Here, we redefine it so that it doesn't appear on the first page if there's a payment coupon (because the coupon will make its own line, at a different position).

\extracouponspace is a config setting for how much space to reserve for the coupon. We will "shorten" the first page's \textheight by that much, which moves the starting position of the header up.

\renewcommand{\headrulewidth}{0pt}
\renewcommand{\footrulewidth}{1pt}

\renewcommand{\footrule}{
[@--
$coupon ? '\ifthenelse{\equal{\thepage}{1}}' : '';
--@]
{
}
{
\vbox to 0pt{\rule{\headwidth}{\footrulewidth}\vss}
}
}

\newcommand{\extracouponspace}{[@-- defined($extracouponspace) ? $extracouponspace : '2.7in' --@]}

Positioning mailing addresses. This is a bad way to do it and I'd recommend using the "textpos" package instead. Also a command here to output the dollar sign (since '$' has special properties in LaTeX). The escaping functions in Template_Mixin.pm know about this, and will substitute \dollar throughout.

% Adjust the inset of the mailing address
\newcommand{\addressinset}[1][]{\hspace{1.0cm}}

% Adjust the inset of the return address and logo
\newcommand{\returninset}[1][]{\hspace{-0.25cm}}

% New command for address lines i.e. skip them if blank
\newcommand{\addressline}[1]{\ifthenelse{\equal{#1}{}}{}{#1\\}}

% Inserts dollar symbol
\newcommand{\dollar}[1][]{\symbol{36}}

=== Headers/footers ===

"fancyhdr" allows separate left, right, and center headers and footers (the first argument to \fancyhead or \fancyfoot). We use left and right headers and a center footer.

These commands are supposed to remove the default "plain" style page footer, but we don't use that page style so it's probably not needed.

% Remove plain style header/footer
\fancypagestyle{plain}{
\fancyhead{}
}
\fancyhf{}

The center footer. On page 1, if there's a coupon, the coupon is the footer. In that case, we move the start position up by \extracouponspace (using \vspace) so that it starts above the normal footer position.

The footer to display below the coupon (or if there isn't one) is configurable as '''invoice_latexfooter'''. The footer to show on subsequent pages is '''invoice_latexsmallfooter'''. By default they're both just the company name.

% Define fancy header/footer for first and subsequent pages
\fancyfoot[C]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
[@--
if ($coupon) {
$OUT .= '\vspace{-\extracouponspace}';
$OUT .= '\rule[0.5em]{\textwidth}{\footrulewidth}\\\\';
$OUT .= $coupon;
$OUT .= '\vspace{'.
(defined($couponfootsep) ? $couponfootsep : '0.2in') .
'}';
}
'';
--@] [@-- $smallerfooter ? '\scriptsize{' : '\small{' --@]
[@-- $footer --@]
}[@-- $coupon ? '\vspace{\extracouponspace}' : '' --@]
}
{ % ... pages
[@-- $smallerfooter ? '\scriptsize{' : '\small{' --@]
[@-- $smallfooter --@]
}
}
}

The right side footer, showing the page number. "emt()" localizes text to the customer's language and escapes for LaTeX. "~" is a non-breaking space character.

\fancyfoot[R]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
}
{ % ... pages
\small{\thepage~[@-- emt('of') --@]~\pageref{LastPage}}
}
}

The left side header, containing the return address and logo on the first page (and nothing on later pages). \returninset is just a horizontal adjustment. \makebox{} tells LaTeX to put its contents into a single box. That's probably unnecessary because the only thing in the box is a table, which is already a single box.

\begin{tabular}{ll} creates a two-column table. In the first column is a "minipage", which is just a fixed-width container environment (5.5cm here) with its bottom edge aligned to the table cell it's in. In here we put the evaluated "returnaddress" template.

Tables in LaTeX always use & to separate columns and \\ to separate rows. A row must always contain as many columns as in the column spec; otherwise LaTeX will guess about what you mean and will usually get it wrong. \\ at the end of the last row is optional.

The second column contains the logo graphic. See "texdoc graphicx" for a full explanation of this command; it can do many useful things including scaling and repositioning the graphic. For example, if you want the logo to be scaled to be 0.5 inch tall, you can do this: "\includegraphics[height=0.5in]{[@-- $logo_file --@]}".

\fancyhead[L]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
\returninset
\makebox{
\begin{tabular}{ll}
\begin{minipage}[b]{5.5cm}
[@-- $returnaddress --@]
\end{minipage} &
\includegraphics{[@-- $logo_file --@]}\\
\end{tabular}
}
}
{ % ... pages
%\includegraphics{[@-- $logo_file --@]} % Uncomment if you want the logo on all pages.
}
}

The right side header, containing the invoice date, invnum, and custnum in a three-column table with each column centered ({ccc}). "$no_date" and "$no_number" are both enabled by arguments to print_generic() and used when printing statements.

Note that "$date" is already formatted using time2str_local('long'), so something like "Feb 1st, 2017". The template can call time2str() to format other dates; this is an alias to time2str_local so it will use the customer's language settings and LaTeX-escape the result.

This is followed by a horizontal line spanning the table (\hline), then a row where the middle cell contains the $notice_name (usually "Invoice") in \huge font, in small caps (\textsc). The \rule{0pt}{5ex} creates a vertical strut (a zero-width rule) which increases the row height to 5ex. Then another \hline.

Note that \hline should never be followed by \\ and will cause errors if it is.

On subsequent pages we show a smaller version without the notice name.

\fancyhead[R]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
\begin{tabular}{ccc}
[@-- join(' & ', ( $no_date ? '' : emt('Invoice date') ),
( $no_number ? '' : emt('Invoice #') ),
emt('Customer #')
)
--@]\\
\vspace{0.2cm}
\textbf{[@-- $date --@]} & \textbf{[@-- $invnum --@]} & \textbf{[@-- $custnum --@]} \\\hline
\rule{0pt}{5ex} &~~ \huge{\textsc{[@-- emt($notice_name) --@]}} & \\
\vspace{-0.2cm}
& & \\\hline
\end{tabular}
}
{ % ... pages
\small{
\begin{tabular}{lll}
[@-- join(' & ', emt('Invoice date'), emt('Invoice #'), emt('Customer #') ) --@]\\
\textbf{[@-- $date --@]} & \textbf{[@-- $invnum --@]} & \textbf{[@-- $custnum --@]}\\
\end{tabular}
}
}
}

Enable the fancyhdr page style and set the font family.

\pagestyle{fancy}

%% Font options are:
%% bch Bitsream Charter
%% put Utopia
%% phv Adobe Helvetica
%% pnc New Century Schoolbook
%% ptm Times
%% pcr Courier

\renewcommand{\familydefault}{phv}

=== Line item table macros ===

This section is Freeside-specific. The main invoice line item table is an 8-column longtable. Normally the columns are:
* Package number
* Description (6 columns)
* Price

(The 6 columns are used separately for showing CDRs, if there are any.)

If '''invoice-unitprice''' is on, it's:
* Package number
* Description (4 columns)
* Unit price
* Quantity
* Price

According to that setting, create macros for how many columns and how much space to allocate to the description, and for the headers of the Unit Price and Unit Quantity columns.

% Commands for freeside table header...

\newcommand{\FSdescriptionlength} { [@-- $unitprices ? '8.2cm' : '12.8cm' --@] }
\newcommand{\FSdescriptioncolumncount} { [@-- $unitprices ? '4' : '6' --@] }
\newcommand{\FSunitcolumns}{ [@--
$unitprices
? '\makebox[2.5cm][r]{\textbf{~~' . emt('Unit Price') . '}} &' .
'\makebox[1.4cm]{\textbf{~' . emt('Quantity') . '}} & '
: '' --@] }

\FShead renders the main table header. Using \makebox inside a column forces it to a minimum width. \multicolumn{N}{X}{ ... } is like COLSPAN in HTML: makes a cell spanning N columns. The second argument is how to align the contents of the column (l, r, or c).

\truncate renders its argument into a space with a maximum width. It will break at word boundaries and insert an ellipsis afterward. Note that if we didn't do this, longtable would either squish the surrounding columns or push them off the page rather than word-wrap the description.

\FSusagehead is the same, but used for usage summary sections that show the number of calls and total minutes. The '''usage_class_summary''' config turns this on.

\newcommand{\FShead}{
\hline
\rule{0pt}{2.5ex}
\makebox[1.4cm]{} &
\multicolumn{\FSdescriptioncolumncount}{l}{
\truncate{\FSdescriptionlength}{\textbf{[@-- emt('Description') --@]}}
} &
\FSunitcolumns
\makebox[1.6cm][r]{\textbf{[@-- emt('Amount') --@]}} \\
\hline
}

\newcommand{\FSusagehead}{
\hline
\rule{0pt}{2.5ex}
\makebox[1.4cm]{} &
\multicolumn{4}{l}{
\truncate{\FSdescriptionlength}{\textbf{[@-- emt('Description') --@]}}
} &
\textbf{~~[@-- emt('Calls') --@]} &
\textbf{~~[@-- emt('Duration') --@]} &
\textbf{~~[@-- emt('Amount') --@]} \\
\hline
}

Next, the commands to actually display line items.

\FSdesc will be called once for each element in the @detail_items array. This takes five arguments (that's the [5] in the declaration) which are the values to put in the pkgnum, description, unit price, quantity, and price columns. If unit prices are off then arguments 3 and 4 will be empty.

\FSextdesc will be called once for each element in the 'ext_description' element of the detail item. These contain things like service labels, discount details, prorate details, and manually created package details. This macro takes a single argument and places it in a big multicolumn below the description, in small font.

\FScalldetail is used only for call details. These are preprocessed by FS::TemplateItem_Mixin::details() ''into LaTeX table rows'' (pre-escaped, delimited with &). So, the macro just skips the first column and outputs its argument.

\FStotaldesc is used for subtotal and total rows; it puts its first argument in a big multicolumn in the description space, and the second in the amount column.

Finally, \FSusagedesc is for usage summary sections, and puts the total calls, duration, and amount in the correct columns.

% ...description...
\newcommand{\FSdesc}[5]{
\multicolumn{1}{c}{\rule{0pt}{2.5ex}\textbf{#1}} &
\multicolumn{[@-- $unitprices ? '4' : '6' --@]}{l}{
\truncate{\FSdescriptionlength}{\textbf{#2}}
} &
[@-- $unitprices ? ' \multicolumn{1}{r}{\textbf{#3}} &'."\n".
' \multicolumn{1}{r}{\textbf{#4}} &'."\n"
: ''
--@]
\multicolumn{1}{r}{\textbf{#5}}\\
}
% ...extended description...
\newcommand{\FSextdesc}[1]{
\multicolumn{1}{l}{\rule{0pt}{1.0ex}} &
\multicolumn{6}{l}{
\truncate{12.8cm}{\small{[[User:Mark|Mark]] ([[User talk:Mark|talk]])#1}}
} \\
}
% ...call detail (multiple columns already)...
\newcommand{\FScalldetail}[1]{
\multicolumn{1}{l}{\rule{0pt}{1.0ex}} &
[[User:Mark|Mark]] ([[User talk:Mark|talk]])#1
\\
}
}
% ...and total line items (which use the full 12.8cm length, ignoring
% unitprice/quantity
\newcommand{\FStotaldesc}[2]{
& \multicolumn{6}{l}{
\truncate{12.8cm}{#1}
} & #2\\
}

% ...usage class summary
\newcommand{\FSusagedesc}[4]{
\multicolumn{1}{c}{\rule{0pt}{2.5ex}} &
\multicolumn{4}{l}{\textbf{#1}} &
\multicolumn{1}{r}{\textbf{#2}} &
\multicolumn{1}{r}{\textbf{#3}} &
\multicolumn{1}{r}{\textbf{#4}}
\\
}

=== Main document ===

First, make a minipage for the customer's name and address. This starts at \addressinset + 5 mm (not sure why) and can be up to 7 cm wide. The \makebox does nothing (as above, a minipage is already a single box). The \addressline macro makes a line break only if the argument is non-empty, to avoid errors for breaking an empty line.

\begin{document}
% Headers and footers defined for the first page
\addressinset \rule{0.5cm}{0cm}
\makebox{
\begin{minipage}[t]{7.0cm}
\vspace{[@-- defined($addresssep) ? $addresssep : '0.25cm' --@]}
\textbf{[@-- $payname --@]}\\
\addressline{[@-- $company --@]}
\addressline{[@-- $address1 --@]}
\addressline{[@-- $address2 --@]}
\addressline{[@-- $city --@], [@-- $state --@]~~[@-- $zip --@]}
\addressline{[@-- $country --@]}
\end{minipage}}

Make another minipage for the service address, terms, and PO number. $ship_enable is true if the global '''invoice-ship_address''' or per-customer "invoice_ship_address" flag is on.

The initial \hfill spreads out the two minipages so that they go all the way to the margins. The 'terms' and 'po_line' should probably be in \addressline{} macros but currently aren't.

After that, skip some vertical space. (If you need more vertical space for an invoice format, the 1.5cm here can be cut.)

\hfill
\makebox{
\begin{minipage}[t]{6.4cm}
[@--
if ($ship_enable) {
$OUT .= '\textbf{' . emt('Service Address') . '}\\\\';
$OUT .= "\\addressline{$ship_company}";
$OUT .= "\\addressline{$ship_address1}";
$OUT .= "\\addressline{$ship_address2}";
$OUT .= "\\addressline{$ship_city, $ship_state~~$ship_zip}";
$OUT .= "\\addressline{$ship_country}";
$OUT .= '~\\\\';
}else{
$OUT .= '';
}
--@]
\begin{flushright}
[@-- $terms ? emt('Terms') . ': ' . emt($terms) : '' --@]\\
[@-- $po_line --@]\\
\end{flushright}
\end{minipage}}
\vspace{1.5cm}

Insert the [[Summary]] if there is one. The last thing in the summary template is a page break.
%
[@-- $summary --@]
%

=== Sections ===

Start creating sections, according to the @sections array. Section entries contain the following keys:

* description
* category (pkg_category.categoryname, for package sections by category)
* location (hashref of location fields, for package sections by location)
* pretotal (formatted with \FStotaldesc, before detail lines)
* subtotal (formatted with \FStotaldesc, at the end)
* posttotal (line to display after the end of the section; used for "Balance Due" message and a few other things)
* sort_weight (sort order, but also does complicated things)

and optionally some flags:
* post_total (put the section at the end of the invoice)
* summarized (omit the section entirely)
* usage_section (use \FSusagedesc instead of \FSdesc, see above)
* no_subtotal (omit the subtotal line)

If '''invoice_sections''' is enabled: the $multisection flag in print_generic() is turned on, and @sections is filled by calling FS::Template_Mixin::_items_sections, which makes a section for each package category. Line items for packages go into the package category's section. '''invoice_sections_method''' can be used to divide sections by location rather than by category. In addition, sections are created for previous invoices, taxes, and "adjustments" (credits and payments to this invoice).

If not: a single section is created, with empty description, containing previous invoices, current charges, and adjustments (after the total). '''previous_balance-section''' will make a previous invoices section anyway.

Additional sections are created in some cases. '''svc_phone_sections''' creates a section for each phone line (see RT#6592; note that this customer is no longer active). '''voip-cust_accountcode_cdr''' creates a section for each accountcode (see RT#12181). '''discount-show_available''' creates a section with information on available discounts. '''usage_class_summary''' creates a section with usage subtotals.

'''finance_pkgclass''' specifies the package class containing "finance charges", which is handled specially. That class's category name is passed to the template as $finance_section, and the sum of charges in that category is passed as $finance_amount. That amount is broken out from the "Current charges" line in the summary section. On a multisection summary-format invoice, the finance charges will also ''not'' be shown in the body of the invoice. I don't know why.

Loop over all sections (except for the finance charges, maybe). Inside the loop we'll do everything by appending to $OUT.

\section*{}
[@--
foreach my $section ( grep { !$summary || $_->{description} ne $finance_section } @sections ) {

We're going to do many things if "!$summary". If there's a summary section, the pretotals and posttotals are redundant so skip them.

if ($section->{'pretotal'} && !$summary) {
$OUT .= '\begin{flushright}';
$OUT .= '\large\textsc{'. $section->{'pretotal'}. '}\\\\';
$OUT .= '\\end{flushright}';
}

Start a new page if this is a "post_total" section. As far as I know these are used only for usage details. I think the 'summarized' flag is no longer used for anything.

The section heading is going to be a caption placed inside the main table. \captionsetup sets properties for the caption (left justified, bold, small caps, Large size). If this is the first page, also tell longtable (via \LTextracouponspace) to shorten the effective length of the page so that it doesn't overprint the coupon.

$OUT .= '\pagebreak' if $section->{'post_total'};
unless ($section->{'summarized'} ) {
$OUT .= '\captionsetup{singlelinecheck=false,justification=raggedright,font={Large,sc,bf}}';
$OUT .= '\ifthenelse{\equal{\thepage}{1}}{\setlength{\LTextracouponspace}{\extracouponspace}}{\setlength{\LTextracouponspace}{0pt}}'
if $coupon;

Start the longtable that will contain the section's line items. Eight columns, pkgnum (centered), description and optional unitprice/quantity (left), then price (right).

\caption* produces a table caption without prefixing it with "Table 1". The caption is the section description, or 'Charges' if it's null (except that if packages are sectioned by location, we put one together from the location fields).

$OUT .= '\begin{longtable}{cllllllr}';
$OUT .= '\caption*{ ';
if ($section->{'location'}) {
$OUT .= $section->{'location'}{'label_prefix'}. ': '
if length($section->{'location'}{'label_prefix'});
$OUT .= $section->{'location'}{'address1'};
$OUT .= ', ' . $section->{'location'}{'address2'}
if length($section->{'location'}{'address2'});
$OUT .= ', ' .
$section->{'location'}{'city'} . ', ' .
$section->{'location'}{'state'} . '~' .
$section->{'location'}{'zip'};
} elsif ( $section->{'description'} ) {
$OUT .= ($section->{'description'});
} else {
$OUT .= emt('Charges');
}
$OUT .= '}\\\\';

The point of a longtable is that it can span multiple pages, including printing head/foot rows at the top/bottom of each page automatically. The protocol for setting these up is to specify the first head, per-page head, per-page foot, and last foot, then the rest of the table contents. Here, we set the first head to be \FShead (or \FSusagehead). \endfirsthead means we're done specifying it.

"header_generator" and other foo_generators are from an old mechanism that injected callbacks into the template data. This is strongly deprecated.

if ($section->{header_generator}) {
$OUT .= &{$section->{header_generator}}();
} elsif ( $section->{usage_section} ) {
$OUT .= '\FSusagehead';
} else {
$OUT .= '\FShead';
}
$OUT .= '\endfirsthead';

The per-page head: a row with a "Continued" message, then \FShead again. The per-page foot is the same.

$OUT .= '\multicolumn{7}{r}{\rule{0pt}{2.5ex}'.emt('Continued from previous page').'}\\\\';
if ($section->{header_generator}) {
$OUT .= &{$section->{header_generator}}();
} elsif ( $section->{usage_section} ) {
$OUT .= '\FSusagehead';
} else {
$OUT .= '\FShead';
}
$OUT .= '\endhead';
$OUT .= '\multicolumn{7}{r}{\rule{0pt}{2.5ex}'.emt('Continued on next page...').'}\\\\';
$OUT .= '\endfoot';
$OUT .= '\hline';

The final foot for the section. For multisection invoices this contains the section subtotal:

if (scalar(@sections) > 1 and !$section->{no_subtotal}) {
if ($section->{total_generator}) {
$OUT .= &{$section->{total_generator}}($section);
} else {
$OUT .= '\FStotaldesc{' . $section->{'description'} . ' Total}' .
'{' . $section->{'subtotal'} . '}' . "\n";
}
}

Single-section invoices use "@total_items" for items that need to appear after the line items. These include the pre-tax subtotal, taxes, total charges, adjustments, and the balance due. Call \FStotaldesc for each of them. Then end the section foot.

#if ($section == $sections[$#sections]) {
foreach my $line (grep {$_->{section}->{description} eq $section->{description}} @total_items) {
if ($section->{total_line_generator}) {
$OUT .= &{$section->{total_line_generator}}($line);
} else {
$OUT .= '\FStotaldesc{' . $line->{'total_item'} . '}' .
'{' . $line->{'total_amount'} . '}' . "\n";
}
}
#}

$OUT .= '\hline';
$OUT .= '\endlastfoot';

Line items are passed in @detail_items. A line item entry can contain:

* section (reference)
* ref (pkgnum)
* description
* ext_description (arrayref of additional lines)
* quantity
* amount
* duration (for usage summary sections)
* pkgpart
* unit_amount (unit price)
* locationnum
* svc_label (the primary service's label)

Line items are created in several places in Template_Mixin and TemplateItem_Mixin, most importantly _items_cust_bill_pkg.

@detail_items is a single array; we find the lines belonging to this section by checking their 'section' element. If there's only one section, process all the lines.

my $lastref = 0;
foreach my $line (
grep { ( scalar( @sections ) > 1
? $section->{'description'} eq $_->{'section'}->{'description'}
: 1
) }
@detail_items )
{
my $ext_description = $line->{'ext_description'};

# Don't break-up small packages.
my $rowbreak = @$ext_description < 5 ? '*' : '';

$lastref is set to the pkgnum of the previous line item. If the value of 'ref' no longer equals that, we're now showing a different package; separate with an \hline. After outputting this line item we'll set $lastref to the new pkgnum.

If this is a usage summary section, convert the duration to minutes/seconds and call \FSusagedesc, passing the description, quantity (number of calls), duration, and total price.

If it's a normal section, call \FSdesc, passing the description, unit price, quantity, and amount. If unit prices aren't in use, or there isn't one on this line item, then leave unit price and quantity blank.

"$rowbreak" is a flag for whether to allow a page break. In general we want to keep line items and their ext_description lines together, unless there are a lot of description lines. So if there are fewer than 5 of them, we set $rowbreak = '*', which after \FSdesc{} is evaluated, results in the table row delimiter being "\\*". This tells LaTeX to avoid putting a page break immediately after this row. Probably we should set $rowbreak = '' on the last description line to explicitly allow page breaks between packages.

$OUT .= "\\hline\n" if (($line->{'ref'} || 0) ne $lastref);
if ($section->{description_generator}) {
$OUT .= &{$section->{description_generator}}($line);
} elsif ($section->{usage_section}) {
my $minutes = sprintf('%d', $line->{'duration'} / 60);
my $seconds = $line->{'duration'} % 60;
$OUT .= '\FSusagedesc
{' . $line->{'description'} . '}
{' . $line->{'quantity'} . '}
{' . $minutes . 'm ' . $seconds . 's' . '}
{' . $line->{'amount'} . '}';
} else {
$OUT .= '\FSdesc'.
'{}'.
'{' . $line->{'description'} . '}' ;
if ( $unitprices and length($line->{'unit_amount'}) ) {
# then show the unit amount and quantity
$OUT .=
'{\\dollar' . $line->{'unit_amount'} . '}'.
'{' . $line->{'quantity'} . '}';
} else {
# leave those columns blank
$OUT .= '{}{}';
}
$OUT .= '{\\dollar' . $line->{'amount'} . "}${rowbreak}\n";
}
$lastref = $line->{'ref'} || 0;

Output ext_description lines. We assume that any of them that contain an unescaped "&" are call details and should use \FScalldetail. Otherwise they use \FSextdesc.

foreach my $ext_desc (@$ext_description) {
if ($section->{extended_description_generator}) {
$OUT .= &{$section->{extended_description_generator}}($ext_desc);
} elsif ( $ext_desc !~ /[^\\]&/ ) {
$OUT .= '\FSextdesc{' . $ext_desc . "}$rowbreak\n";
} else { # call detail
$OUT .= '\FScalldetail{' . $ext_desc . "}$rowbreak\n";
}
}

}

End the longtable. If there is a posttotal item in this section (the final Balance Due / Credit Balance message, previous invoice aging, a few other things) then show it right-justified, in the same formatting used for the caption.

$OUT .= '\end{longtable}';
}
if ($section->{'posttotal'}) {
$OUT .= '\begin{flushright}';
$OUT .= '\normalfont\large\bfseries\textsc{'. $section->{'posttotal'}. '}\\\\';
$OUT .= '\\end{flushright}';
}
}

=== Notes ===

For the notes panel, create a minipage the full width of the text. (Unless $summary is in use; then the notes were on the summary page.)

There's some tricky handling of vertical space here. We want the notes at the bottom of the page, regardless of where the last section ended, so the \vfill between them acts as an expandable space.

However, if the notes end up on the first page, we have to prevent them from overprinting the coupon. So if there's a coupon and the notes are on page 1, we insert a strut of height \extracouponspace below the notes. The \vfill will then expand so that the bottom of the ''strut'' is at the bottom of the page, keeping the coupon space clear.

--@]
\vfill
\begin{minipage}[t]{\textwidth}
[@-- length($summary)
? ''
: ( $smallernotes
? '\scriptsize{ '.$notes.' }'
: $notes
)
--@]
[@-- $coupon ? '\ifthenelse{\equal{\thepage}{1}}{\rule{0pt}{\extracouponspace}}{}' : '' --@]
\end{minipage}
\end{document}

That's the end of the invoice.

== The summary template ==

Start with a two-column table, left side for the notes and right side for the billing summary. On the left side, make a 6.4cm minipage, and put a 10cm vertical strut there, along with the notes.

\begin{tabular}{ll}
\begin{minipage}{6.4cm}
\begin{tabular}{m{0cm}m{6.4cm}}
\rule{0cm}{10cm}&\begin{minipage}{6cm}[@-- $notes --@]\end{minipage}\\
\end{tabular}
\end{minipage} &

On the right side, place a 2cm horizontal strut followed by a 12.8cm minipage. This will contain the billing summary.

\rule{2cm}{0cm}
\begin{minipage}{12.8cm}

The billing summary itself is a two-column table, one for labels and one for amounts. The section headings are just labels with no amounts, with lines underneath.

'true_previous_balance' is, as far as we can determine it, the balance that was printed on the customer's previous invoice. It's the sum of all the customer's transactions (invoices, credits, payments, refunds) dated before that invoice, plus the invoice amount.

'balance_adjustments' is the sum of payments and credits received after the previous invoice, but applied to invoices before this current invoice. 'true_previous_balance' - 'balance_adjustments' is the total amount owed on all past invoices.

\begin{tabular}{lr}
\hline
&\\
\textbf{\underline{Summary of Previous Balance and Payments}} & \\
&\\
\textbf{Previous Balance}&\textbf{\dollar[@-- $true_previous_balance --@]}\\
\textbf{Payments}&\textbf{\dollar[@-- $balance_adjustments --@]}\\
\cline{2-2}
\textbf{Balance Outstanding}&\textbf{\dollar[@-- sprintf('%.2f', $true_previous_balance -$balance_adjustments) --@]}\\
&\\
\hline
&\\
\textbf{\underline{Summary of New Charges}} & \\
&\\

@summary_subtotals contains one line for each invoice section, except for the sections that have special handling such as the adjustments and finance charges. Make a line for each of those subtotals. This is followed with \cline{2-2}, which puts a horizontal line under the second column only, and then a row for the total of new charges.

[@--
foreach my $section (@summary_subtotals) {
$OUT .= '\textbf{'. ($section->{'description'} ? $section->{'description'} : 'Charges' ). '}';
$OUT .= '&\textbf{'. $section->{'subtotal'}. '}\\\\';
}
$OUT .= '\cline{2-2}';
--@]
\textbf{New Charges Total}&\textbf{\dollar[@-- $current_less_finance --@]}\\
&\\
\hline
&\\
\textbf{\underline{Invoice Summary}} & \\
& \\
\textbf{Previous Past Due Charges}&\textbf{\dollar[@-- sprintf('%.2f', $true_previous_balance - $balance_adjustments) --@]}\\
\textbf{Finance charges on overdue amount}&\textbf{\dollar[@-- $finance_amount --@]}\\
\textbf{New Charges}&\textbf{\dollar[@-- $current_less_finance --@]}\\

Then show the subtotal of the adjustment section, if there is one. This contains payments and credits applied to the ''current'' invoice.

[@--
#false laziness w/invoice_htmlsummary and above
foreach my $section ( grep $_->{adjust_section}, @sections ) {
$OUT .= '\textbf{'. ($section->{'description'} ? $section->{'description'} : 'Charges' ). '}';
$OUT .= '&\textbf{'. $section->{'subtotal'}. '}\\\\';
}
--@]

Finally, the customer's balance including the current invoice, and then close the table and minipage, and force a page break so the line items can start on a new page.

\cline{2-2}
\textbf{Total Amount Due}&\textbf{\dollar[@-- sprintf('%.2f', $balance) --@]}\\
&\\
\hline
\end{tabular}
\end{minipage} \\
\end{tabular}
\newpage

== Invoice modes ==

Invoice template documentation

2017-01-10T22:01:57Z

Mark: Created page with "= General notes = Templates for PDF invoices are in LaTeX, with Perl inclusions in [@-- --@] blocks. Each block can insert a string at its position by either assigning the st..."

= General notes =

Templates for PDF invoices are in LaTeX, with Perl inclusions in [@-- --@] blocks. Each block can insert a string at its position by either assigning the string to the $OUT variable, or simply returning a string. For details, see Text::Template.

The main function for generating the LaTeX code is FS::Template_Mixin::print_generic. This is a method of the cust_bill object (and also of quotation objects), and takes several arguments described in the perldoc. Most of what this function does is set up the "%invoice_data" hash, which gets imported into local variable space when evaluating the template.

print_generic() actually constructs several templates: 'notes', 'coupon', 'footer', 'smallfooter', 'watermark', and sometimes 'summary'. Exactly which config variable is loaded for each of these is decided in Template_Mixin; look for the line that starts "my $tc". It's '''invoice_$format$part''' (where $format is either "latex" or "html" and $part is "notes", "coupon", etc.). Each template is evaluated, with %invoice_data as arguments, then put back into %invoice_data as "$notes", "$coupon", etc. The main template contains code fragments to include them.

To further complicate things, all of these elements have locale overrides AND invoice mode overrides. If print_generic() was called with a 'mode' argument, those templates will be fetched from the invoice mode + customer locale. If not, they'll be taken from the system config for the customer's agent and locale.

You can obtain the generated LaTeX code for a particular invoice with ''view/cust_bill-tex.cgi'', which takes all the same parameters as ''view/cust_bill.cgi''.

The template mechanics make it difficult to insert graphics other than the logo. During the rendering process, the content of the logo.eps config is written to a file with a random name, and then that name is passed as the 'logo_file' parameter. However, the \includegraphics command has access to the entire filesystem, so one option is to place static graphics somewhere under /home/freeside.

One useful tool for editing a template is Overleaf (https://www.overleaf.com/), which is a realtime LaTeX editor. This will require you to upload the logo EPS file with the correct randomly generated name (or edit the references to it) along with the modified longtable.sty file. Also, since it can't evaluate the Perl blocks, you have to work with the generated code rather than the template itself, and then manually edit your changes back into the template. It's still much easier than the cycle of "edit template, set config, download and examine PDF", especially since it will display LaTeX errors in a semi-useful way.

== Positioning ==

This is not used in the standard template but has been used in custom templates. Often in an invoice there's a need to place something at a known position on the page, like a return address that needs to line up with an envelope window. LaTeX isn't designed to facilitate that but LaTeX can do absolutely anything, so: the "textpos" package.

"textpos" requires you to set up a grid system, like this:

\usepackage[absolute]{textpos}
\setlength{\TPHorizModule}{1in}
\setlength{\TPVertModule}{1in}

The environment for a fixed position on the page is "textblock". The syntax is odd; notice parentheses rather than braces, and numbers without units.

\begin{textblock}{3.25}(0.625, 0.5)
\footnotesize{
This text is in a box starting 0.625in left and 0.5in down from the page
origin, in a box 3.25in wide. It will fit inside an envelope window.
}
\end{textblock}

The textblock behaves a lot like a minipage, and can contain almost anything; in particular, \includegraphics works. It's positioned after everything else on the page, which means there's no protection against overprinting anything.

== The standard invoice template ==

=== Preamble ===

Set up the document class and paper size, load all packages we need. In general, all \usepackage commands should be at the start of the document.

* "fancyhdr" supplies the \fancyhead and \fancyfoot commands to place left, right, and center headers and footers.
* "lastpage" creates the LastPage reference for displaying "page X of Y" labels.
* "ifthen" provides the \ifthenelse command.
* "array" provides support for custom table column formats.
* "longtable" provides tables that can span multiple pages. We ship a modified longtable.sty, which reserves vertical space on the first page to allow for the coupon. The amount of vertical space is in the "\LTcouponspace" variable.
* "afterpage" was removed from the template in May 2005 and we can remove the package at some point.
* "multirow" is used to make a table cell in the coupon to hold the return address.
* "bigstrut" appears not to be used.
* "truncate" provides a command to apply a width limit to a line of text, so that anything past that gets replaced with an ellipsis. This is highly encouraged for things like package descriptions that could otherwise overflow their table cells.
* "graphicx" provides the \includegraphics command for inserting the EPS of the logo.
* "inputenc" and "fontenc" make LaTeX play nicely with UTF-8 characters.
* "background" is used to create watermarks (such as are used for some past-due notices, or voided invoices). The \backgroundsetup command takes a TikZ argument list, which is extremely powerful. See "texdoc pgf" for all the horrifying details.

Useful things to add here:

* "textpos": see above.
* "geometry" to manage the margins and headers in a centralized way. It also takes the [showframe] option, which will draw lines to visualize the page layout.
* "xcolor" if you want color.
* "tabularx" and/or "tabulary" add more options for setting table column widths.

\documentclass[letterpaper]{article}

\usepackage{fancyhdr,lastpage,ifthen,array,longtable,afterpage,caption,multirow,bigstrut}
\usepackage[breakwords]{truncate} % to avoid overflowing boxes
\usepackage{graphicx} % required for logo graphic
\usepackage[utf8]{inputenc} % multilanguage support
\usepackage[T1]{fontenc}
[@-- if ( length($watermark) ) {
$OUT .= '
\usepackage{background}
\backgroundsetup{
placement=center,
opacity=0.25,
color=black,
angle=0,
contents=' . $watermark . '
}';
}
'';
--@]

Set up dimensions. Most of these were determined empirically and really ought to use the "geometry" package. A few of the dimensions can be adjusted by config variables: \topmargin, \headsep, \textheight.

See [https://en.wikibooks.org/wiki/LaTeX/Page_Layout] and [http://mirrors.ctan.org/macros/latex/base/letter.pdf texdoc letter] for how LaTeX thinks a letter-class page is structured. Note that everything is laid out statically from the top of the page. The header starts at (1in + \voffset + \topmargin), and is placed in a box with height \headheight, and then the document body starts at (1in + \voffset + \topmargin + \headheight + \headsep), and then the footer starts (\textheight + \footskip) below that.

If the header doesn't fit within \headheight, LaTeX will complain, and then adjust the headheight, pushing everything else down. There's also not actually any space reserved for the footer; it has to fit into the space left below the body container.

Commands for manipulating length variables: \newlength declares them (and all of these are predeclared in the document stylesheet); \setlength sets them to a value; \addtolength adds to them.

If you're adjusting margins, it can be helpful to add "\usepackage{layout}" to the preamble, and then "\newpage\layout" before the final "\end{document}". This will print a diagram showing the layout boxes and their dimensions.

\LTchunksize is an internal variable in longtable for how many rows to process at a time. There's probably no reason to change it.

\addtolength{\voffset}{-0.0cm} % top margin to top of header
\addtolength{\hoffset}{-0.6cm} % left margin on page
\addtolength{\topmargin}{[@-- defined($topmargin) ? $topmargin : '-1.00cm' --@]}
\setlength{\headheight}{2.0cm} % height of header
\setlength{\headsep}{[@-- defined($headsep) ? $headsep : '1.0cm' --@]}
\setlength{\footskip}{1.0cm} % bottom of footer from bottom of text

%\addtolength{\textwidth}{2.1in} % width of text
\setlength{\textwidth}{19.5cm}
\setlength{\textheight}{[@-- defined($textheight) ? $textheight : '19.5cm' --@]}
\setlength{\oddsidemargin}{-0.9cm} % odd page left margin
\setlength{\evensidemargin}{-0.9cm} % even page left margin

\LTchunksize=40

$coupon is a local variable containing the entire generated content of the [[Invoice_coupon_template|invoice payment coupon]]. If for some reason there isn't a payment coupon (the customer doesn't owe any money, or this is a quotation or statement rather than an invoice) then it's an empty string.

\footrule is the macro used to draw a line at the top of the footer. Here, we redefine it so that it doesn't appear on the first page if there's a payment coupon (because the coupon will make its own line, at a different position).

\extracouponspace is a config setting for how much space to reserve for the coupon. We will "shorten" the first page's \textheight by that much, which moves the starting position of the header up.

\renewcommand{\headrulewidth}{0pt}
\renewcommand{\footrulewidth}{1pt}

\renewcommand{\footrule}{
[@--
$coupon ? '\ifthenelse{\equal{\thepage}{1}}' : '';
--@]
{
}
{
\vbox to 0pt{\rule{\headwidth}{\footrulewidth}\vss}
}
}

\newcommand{\extracouponspace}{[@-- defined($extracouponspace) ? $extracouponspace : '2.7in' --@]}

Positioning mailing addresses. This is a bad way to do it and I'd recommend using the "textpos" package instead. Also a command here to output the dollar sign (since '$' has special properties in LaTeX). The escaping functions in Template_Mixin.pm know about this, and will substitute \dollar throughout.

% Adjust the inset of the mailing address
\newcommand{\addressinset}[1][]{\hspace{1.0cm}}

% Adjust the inset of the return address and logo
\newcommand{\returninset}[1][]{\hspace{-0.25cm}}

% New command for address lines i.e. skip them if blank
\newcommand{\addressline}[1]{\ifthenelse{\equal{#1}{}}{}{#1\\}}

% Inserts dollar symbol
\newcommand{\dollar}[1][]{\symbol{36}}

=== Headers/footers ===

"fancyhdr" allows separate left, right, and center headers and footers (the first argument to \fancyhead or \fancyfoot). We use left and right headers and a center footer.

These commands are supposed to remove the default "plain" style page footer, but we don't use that page style so it's probably not needed.

% Remove plain style header/footer
\fancypagestyle{plain}{
\fancyhead{}
}
\fancyhf{}

The center footer. On page 1, if there's a coupon, the coupon is the footer. In that case, we move the start position up by \extracouponspace (using \vspace) so that it starts above the normal footer position.

The footer to display below the coupon (or if there isn't one) is configurable as '''invoice_latexfooter'''. The footer to show on subsequent pages is '''invoice_latexsmallfooter'''. By default they're both just the company name.

% Define fancy header/footer for first and subsequent pages
\fancyfoot[C]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
[@--
if ($coupon) {
$OUT .= '\vspace{-\extracouponspace}';
$OUT .= '\rule[0.5em]{\textwidth}{\footrulewidth}\\\\';
$OUT .= $coupon;
$OUT .= '\vspace{'.
(defined($couponfootsep) ? $couponfootsep : '0.2in') .
'}';
}
'';
--@] [@-- $smallerfooter ? '\scriptsize{' : '\small{' --@]
[@-- $footer --@]
}[@-- $coupon ? '\vspace{\extracouponspace}' : '' --@]
}
{ % ... pages
[@-- $smallerfooter ? '\scriptsize{' : '\small{' --@]
[@-- $smallfooter --@]
}
}
}

The right side footer, showing the page number. "emt()" localizes text to the customer's language and escapes for LaTeX. "~" is a non-breaking space character.

\fancyfoot[R]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
}
{ % ... pages
\small{\thepage~[@-- emt('of') --@]~\pageref{LastPage}}
}
}

The left side header, containing the return address and logo on the first page (and nothing on later pages). \returninset is just a horizontal adjustment. \makebox{} tells LaTeX to put its contents into a single box. That's probably unnecessary because the only thing in the box is a table, which is already a single box.

\begin{tabular}{ll} creates a two-column table. In the first column is a "minipage", which is just a fixed-width container environment (5.5cm here) with its bottom edge aligned to the table cell it's in. In here we put the evaluated "returnaddress" template.

Tables in LaTeX always use & to separate columns and \\ to separate rows. A row must always contain as many columns as in the column spec; otherwise LaTeX will guess about what you mean and will usually get it wrong. \\ at the end of the last row is optional.

The second column contains the logo graphic. See "texdoc graphicx" for a full explanation of this command; it can do many useful things including scaling and repositioning the graphic. For example, if you want the logo to be scaled to be 0.5 inch tall, you can do this: "\includegraphics[height=0.5in]{[@-- $logo_file --@]}".

\fancyhead[L]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
\returninset
\makebox{
\begin{tabular}{ll}
\begin{minipage}[b]{5.5cm}
[@-- $returnaddress --@]
\end{minipage} &
\includegraphics{[@-- $logo_file --@]}\\
\end{tabular}
}
}
{ % ... pages
%\includegraphics{[@-- $logo_file --@]} % Uncomment if you want the logo on all pages.
}
}

The right side header, containing the invoice date, invnum, and custnum in a three-column table with each column centered ({ccc}). "$no_date" and "$no_number" are both enabled by arguments to print_generic() and used when printing statements.

Note that "$date" is already formatted using time2str_local('long'), so something like "Feb 1st, 2017". The template can call time2str() to format other dates; this is an alias to time2str_local so it will use the customer's language settings and LaTeX-escape the result.

This is followed by a horizontal line spanning the table (\hline), then a row where the middle cell contains the $notice_name (usually "Invoice") in \huge font, in small caps (\textsc). The \rule{0pt}{5ex} creates a vertical strut (a zero-width rule) which increases the row height to 5ex. Then another \hline.

Note that \hline should never be followed by \\ and will cause errors if it is.

On subsequent pages we show a smaller version without the notice name.

\fancyhead[R]{
\ifthenelse{\equal{\thepage}{1}}
{ % First page
\begin{tabular}{ccc}
[@-- join(' & ', ( $no_date ? '' : emt('Invoice date') ),
( $no_number ? '' : emt('Invoice #') ),
emt('Customer #')
)
--@]\\
\vspace{0.2cm}
\textbf{[@-- $date --@]} & \textbf{[@-- $invnum --@]} & \textbf{[@-- $custnum --@]} \\\hline
\rule{0pt}{5ex} &~~ \huge{\textsc{[@-- emt($notice_name) --@]}} & \\
\vspace{-0.2cm}
& & \\\hline
\end{tabular}
}
{ % ... pages
\small{
\begin{tabular}{lll}
[@-- join(' & ', emt('Invoice date'), emt('Invoice #'), emt('Customer #') ) --@]\\
\textbf{[@-- $date --@]} & \textbf{[@-- $invnum --@]} & \textbf{[@-- $custnum --@]}\\
\end{tabular}
}
}
}

Enable the fancyhdr page style and set the font family.

\pagestyle{fancy}

%% Font options are:
%% bch Bitsream Charter
%% put Utopia
%% phv Adobe Helvetica
%% pnc New Century Schoolbook
%% ptm Times
%% pcr Courier

\renewcommand{\familydefault}{phv}

=== Line item table macros ===

This section is Freeside-specific. The main invoice line item table is an 8-column longtable. Normally the columns are:
* Package number
* Description (6 columns)
* Price

(The 6 columns are used separately for showing CDRs, if there are any.)

If '''invoice-unitprice''' is on, it's:
* Package number
* Description (4 columns)
* Unit price
* Quantity
* Price

According to that setting, create macros for how many columns and how much space to allocate to the description, and for the headers of the Unit Price and Unit Quantity columns.

% Commands for freeside table header...

\newcommand{\FSdescriptionlength} { [@-- $unitprices ? '8.2cm' : '12.8cm' --@] }
\newcommand{\FSdescriptioncolumncount} { [@-- $unitprices ? '4' : '6' --@] }
\newcommand{\FSunitcolumns}{ [@--
$unitprices
? '\makebox[2.5cm][r]{\textbf{~~' . emt('Unit Price') . '}} &' .
'\makebox[1.4cm]{\textbf{~' . emt('Quantity') . '}} & '
: '' --@] }

\FShead renders the main table header. Using \makebox inside a column forces it to a minimum width. \multicolumn{N}{X}{ ... } is like COLSPAN in HTML: makes a cell spanning N columns. The second argument is how to align the contents of the column (l, r, or c).

\truncate renders its argument into a space with a maximum width. It will break at word boundaries and insert an ellipsis afterward. Note that if we didn't do this, longtable would either squish the surrounding columns or push them off the page rather than word-wrap the description.

\FSusagehead is the same, but used for usage summary sections that show the number of calls and total minutes. The '''usage_class_summary''' config turns this on.

\newcommand{\FShead}{
\hline
\rule{0pt}{2.5ex}
\makebox[1.4cm]{} &
\multicolumn{\FSdescriptioncolumncount}{l}{
\truncate{\FSdescriptionlength}{\textbf{[@-- emt('Description') --@]}}
} &
\FSunitcolumns
\makebox[1.6cm][r]{\textbf{[@-- emt('Amount') --@]}} \\
\hline
}

\newcommand{\FSusagehead}{
\hline
\rule{0pt}{2.5ex}
\makebox[1.4cm]{} &
\multicolumn{4}{l}{
\truncate{\FSdescriptionlength}{\textbf{[@-- emt('Description') --@]}}
} &
\textbf{~~[@-- emt('Calls') --@]} &
\textbf{~~[@-- emt('Duration') --@]} &
\textbf{~~[@-- emt('Amount') --@]} \\
\hline
}

Next, the commands to actually display line items.

\FSdesc will be called once for each element in the @detail_items array. This takes five arguments (that's the [5] in the declaration) which are the values to put in the pkgnum, description, unit price, quantity, and price columns. If unit prices are off then arguments 3 and 4 will be empty.

\FSextdesc will be called once for each element in the 'ext_description' element of the detail item. These contain things like service labels, discount details, prorate details, and manually created package details. This macro takes a single argument and places it in a big multicolumn below the description, in small font.

\FScalldetail is used only for call details. These are preprocessed by FS::TemplateItem_Mixin::details() ''into LaTeX table rows'' (pre-escaped, delimited with &). So, the macro just skips the first column and outputs its argument.

\FStotaldesc is used for subtotal and total rows; it puts its first argument in a big multicolumn in the description space, and the second in the amount column.

Finally, \FSusagedesc is for usage summary sections, and puts the total calls, duration, and amount in the correct columns.

% ...description...
\newcommand{\FSdesc}[5]{
\multicolumn{1}{c}{\rule{0pt}{2.5ex}\textbf{#1}} &
\multicolumn{[@-- $unitprices ? '4' : '6' --@]}{l}{
\truncate{\FSdescriptionlength}{\textbf{#2}}
} &
[@-- $unitprices ? ' \multicolumn{1}{r}{\textbf{#3}} &'."\n".
' \multicolumn{1}{r}{\textbf{#4}} &'."\n"
: ''
--@]
\multicolumn{1}{r}{\textbf{#5}}\\
}
% ...extended description...
\newcommand{\FSextdesc}[1]{
\multicolumn{1}{l}{\rule{0pt}{1.0ex}} &
\multicolumn{6}{l}{
\truncate{12.8cm}{\small{[[User:Mark|Mark]] ([[User talk:Mark|talk]])#1}}
} \\
}
% ...call detail (multiple columns already)...
\newcommand{\FScalldetail}[1]{
\multicolumn{1}{l}{\rule{0pt}{1.0ex}} &
[[User:Mark|Mark]] ([[User talk:Mark|talk]])#1
\\
}
}
% ...and total line items (which use the full 12.8cm length, ignoring
% unitprice/quantity
\newcommand{\FStotaldesc}[2]{
& \multicolumn{6}{l}{
\truncate{12.8cm}{#1}
} & #2\\
}

% ...usage class summary
\newcommand{\FSusagedesc}[4]{
\multicolumn{1}{c}{\rule{0pt}{2.5ex}} &
\multicolumn{4}{l}{\textbf{#1}} &
\multicolumn{1}{r}{\textbf{#2}} &
\multicolumn{1}{r}{\textbf{#3}} &
\multicolumn{1}{r}{\textbf{#4}}
\\
}

=== Main document ===

First, make a minipage for the customer's name and address. This starts at \addressinset + 5 mm (not sure why) and can be up to 7 cm wide. The \makebox does nothing (as above, a minipage is already a single box). The \addressline macro makes a line break only if the argument is non-empty, to avoid errors for breaking an empty line.

\begin{document}
% Headers and footers defined for the first page
\addressinset \rule{0.5cm}{0cm}
\makebox{
\begin{minipage}[t]{7.0cm}
\vspace{[@-- defined($addresssep) ? $addresssep : '0.25cm' --@]}
\textbf{[@-- $payname --@]}\\
\addressline{[@-- $company --@]}
\addressline{[@-- $address1 --@]}
\addressline{[@-- $address2 --@]}
\addressline{[@-- $city --@], [@-- $state --@]~~[@-- $zip --@]}
\addressline{[@-- $country --@]}
\end{minipage}}

Make another minipage for the service address, terms, and PO number. $ship_enable is true if the global '''invoice-ship_address''' or per-customer "invoice_ship_address" flag is on.

The initial \hfill spreads out the two minipages so that they go all the way to the margins. The 'terms' and 'po_line' should probably be in \addressline{} macros but currently aren't.

After that, skip some vertical space. (If you need more vertical space for an invoice format, the 1.5cm here can be cut.)

\hfill
\makebox{
\begin{minipage}[t]{6.4cm}
[@--
if ($ship_enable) {
$OUT .= '\textbf{' . emt('Service Address') . '}\\\\';
$OUT .= "\\addressline{$ship_company}";
$OUT .= "\\addressline{$ship_address1}";
$OUT .= "\\addressline{$ship_address2}";
$OUT .= "\\addressline{$ship_city, $ship_state~~$ship_zip}";
$OUT .= "\\addressline{$ship_country}";
$OUT .= '~\\\\';
}else{
$OUT .= '';
}
--@]
\begin{flushright}
[@-- $terms ? emt('Terms') . ': ' . emt($terms) : '' --@]\\
[@-- $po_line --@]\\
\end{flushright}
\end{minipage}}
\vspace{1.5cm}

Insert the [[Summary]] if there is one. The last thing in the summary template is a page break.
%
[@-- $summary --@]
%

=== Sections ===

Start creating sections, according to the @sections array. Section entries contain the following keys:

* description
* category (pkg_category.categoryname, for package sections by category)
* location (hashref of location fields, for package sections by location)
* pretotal (formatted with \FStotaldesc, before detail lines)
* subtotal (formatted with \FStotaldesc, at the end)
* posttotal (line to display after the end of the section; used for "Balance Due" message and a few other things)
* sort_weight (sort order, but also does complicated things)

and optionally some flags:
* post_total (put the section at the end of the invoice)
* summarized (omit the section entirely)
* usage_section (use \FSusagedesc instead of \FSdesc, see above)
* no_subtotal (omit the subtotal line)

If '''invoice_sections''' is enabled: the $multisection flag in print_generic() is turned on, and @sections is filled by calling FS::Template_Mixin::_items_sections, which makes a section for each package category. Line items for packages go into the package category's section. '''invoice_sections_method''' can be used to divide sections by location rather than by category. In addition, sections are created for previous invoices, taxes, and "adjustments" (credits and payments to this invoice).

If not: a single section is created, with empty description, containing previous invoices, current charges, and adjustments (after the total). '''previous_balance-section''' will make a previous invoices section anyway.

Additional sections are created in some cases. '''svc_phone_sections''' creates a section for each phone line (see RT#6592; note that this customer is no longer active). '''voip-cust_accountcode_cdr''' creates a section for each accountcode (see RT#12181). '''discount-show_available''' creates a section with information on available discounts. '''usage_class_summary''' creates a section with usage subtotals.

'''finance_pkgclass''' specifies the package class containing "finance charges", which is handled specially. That class's category name is passed to the template as $finance_section, and the sum of charges in that category is passed as $finance_amount. That amount is broken out from the "Current charges" line in the summary section. On a multisection summary-format invoice, the finance charges will also ''not'' be shown in the body of the invoice. I don't know why.

Loop over all sections (except for the finance charges, maybe). Inside the loop we'll do everything by appending to $OUT.

\section*{}
[@--
foreach my $section ( grep { !$summary || $_->{description} ne $finance_section } @sections ) {

We're going to do many things if "!$summary". If there's a summary section, the pretotals and posttotals are redundant so skip them.

if ($section->{'pretotal'} && !$summary) {
$OUT .= '\begin{flushright}';
$OUT .= '\large\textsc{'. $section->{'pretotal'}. '}\\\\';
$OUT .= '\\end{flushright}';
}

Start a new page if this is a "post_total" section. As far as I know these are used only for usage details. I think the 'summarized' flag is no longer used for anything.

The section heading is going to be a caption placed inside the main table. \captionsetup sets properties for the caption (left justified, bold, small caps, Large size). If this is the first page, also tell longtable (via \LTextracouponspace) to shorten the effective length of the page so that it doesn't overprint the coupon.

$OUT .= '\pagebreak' if $section->{'post_total'};
unless ($section->{'summarized'} ) {
$OUT .= '\captionsetup{singlelinecheck=false,justification=raggedright,font={Large,sc,bf}}';
$OUT .= '\ifthenelse{\equal{\thepage}{1}}{\setlength{\LTextracouponspace}{\extracouponspace}}{\setlength{\LTextracouponspace}{0pt}}'
if $coupon;

Start the longtable that will contain the section's line items. Eight columns, pkgnum (centered), description and optional unitprice/quantity (left), then price (right).

\caption* produces a table caption without prefixing it with "Table 1". The caption is the section description, or 'Charges' if it's null (except that if packages are sectioned by location, we put one together from the location fields).

$OUT .= '\begin{longtable}{cllllllr}';
$OUT .= '\caption*{ ';
if ($section->{'location'}) {
$OUT .= $section->{'location'}{'label_prefix'}. ': '
if length($section->{'location'}{'label_prefix'});
$OUT .= $section->{'location'}{'address1'};
$OUT .= ', ' . $section->{'location'}{'address2'}
if length($section->{'location'}{'address2'});
$OUT .= ', ' .
$section->{'location'}{'city'} . ', ' .
$section->{'location'}{'state'} . '~' .
$section->{'location'}{'zip'};
} elsif ( $section->{'description'} ) {
$OUT .= ($section->{'description'});
} else {
$OUT .= emt('Charges');
}
$OUT .= '}\\\\';

The point of a longtable is that it can span multiple pages, including printing head/foot rows at the top/bottom of each page automatically. The protocol for setting these up is to specify the first head, per-page head, per-page foot, and last foot, then the rest of the table contents. Here, we set the first head to be \FShead (or \FSusagehead). \endfirsthead means we're done specifying it.

"header_generator" and other foo_generators are from an old mechanism that injected callbacks into the template data. This is strongly deprecated.

if ($section->{header_generator}) {
$OUT .= &{$section->{header_generator}}();
} elsif ( $section->{usage_section} ) {
$OUT .= '\FSusagehead';
} else {
$OUT .= '\FShead';
}
$OUT .= '\endfirsthead';

The per-page head: a row with a "Continued" message, then \FShead again. The per-page foot is the same.

$OUT .= '\multicolumn{7}{r}{\rule{0pt}{2.5ex}'.emt('Continued from previous page').'}\\\\';
if ($section->{header_generator}) {
$OUT .= &{$section->{header_generator}}();
} elsif ( $section->{usage_section} ) {
$OUT .= '\FSusagehead';
} else {
$OUT .= '\FShead';
}
$OUT .= '\endhead';
$OUT .= '\multicolumn{7}{r}{\rule{0pt}{2.5ex}'.emt('Continued on next page...').'}\\\\';
$OUT .= '\endfoot';
$OUT .= '\hline';

The final foot for the section. For multisection invoices this contains the section subtotal:

if (scalar(@sections) > 1 and !$section->{no_subtotal}) {
if ($section->{total_generator}) {
$OUT .= &{$section->{total_generator}}($section);
} else {
$OUT .= '\FStotaldesc{' . $section->{'description'} . ' Total}' .
'{' . $section->{'subtotal'} . '}' . "\n";
}
}

Single-section invoices use "@total_items" for items that need to appear after the line items. These include the pre-tax subtotal, taxes, total charges, adjustments, and the balance due. Call \FStotaldesc for each of them. Then end the section foot.

#if ($section == $sections[$#sections]) {
foreach my $line (grep {$_->{section}->{description} eq $section->{description}} @total_items) {
if ($section->{total_line_generator}) {
$OUT .= &{$section->{total_line_generator}}($line);
} else {
$OUT .= '\FStotaldesc{' . $line->{'total_item'} . '}' .
'{' . $line->{'total_amount'} . '}' . "\n";
}
}
#}

$OUT .= '\hline';
$OUT .= '\endlastfoot';

Line items are passed in @detail_items. A line item entry can contain:

* section (reference)
* ref (pkgnum)
* description
* ext_description (arrayref of additional lines)
* quantity
* amount
* duration (for usage summary sections)
* pkgpart
* unit_amount (unit price)
* locationnum
* svc_label (the primary service's label)

Line items are created in several places in Template_Mixin and TemplateItem_Mixin, most importantly _items_cust_bill_pkg.

@detail_items is a single array; we find the lines belonging to this section by checking their 'section' element. If there's only one section, process all the lines.

my $lastref = 0;
foreach my $line (
grep { ( scalar( @sections ) > 1
? $section->{'description'} eq $_->{'section'}->{'description'}
: 1
) }
@detail_items )
{
my $ext_description = $line->{'ext_description'};

# Don't break-up small packages.
my $rowbreak = @$ext_description < 5 ? '*' : '';

$lastref is set to the pkgnum of the previous line item. If the value of 'ref' no longer equals that, we're now showing a different package; separate with an \hline. After outputting this line item we'll set $lastref to the new pkgnum.

If this is a usage summary section, convert the duration to minutes/seconds and call \FSusagedesc, passing the description, quantity (number of calls), duration, and total price.

If it's a normal section, call \FSdesc, passing the description, unit price, quantity, and amount. If unit prices aren't in use, or there isn't one on this line item, then leave unit price and quantity blank.

"$rowbreak" is a flag for whether to allow a page break. In general we want to keep line items and their ext_description lines together, unless there are a lot of description lines. So if there are fewer than 5 of them, we set $rowbreak = '*', which after \FSdesc{} is evaluated, results in the table row delimiter being "\\*". This tells LaTeX to avoid putting a page break immediately after this row. Probably we should set $rowbreak = '' on the last description line to explicitly allow page breaks between packages.

$OUT .= "\\hline\n" if (($line->{'ref'} || 0) ne $lastref);
if ($section->{description_generator}) {
$OUT .= &{$section->{description_generator}}($line);
} elsif ($section->{usage_section}) {
my $minutes = sprintf('%d', $line->{'duration'} / 60);
my $seconds = $line->{'duration'} % 60;
$OUT .= '\FSusagedesc
{' . $line->{'description'} . '}
{' . $line->{'quantity'} . '}
{' . $minutes . 'm ' . $seconds . 's' . '}
{' . $line->{'amount'} . '}';
} else {
$OUT .= '\FSdesc'.
'{}'.
'{' . $line->{'description'} . '}' ;
if ( $unitprices and length($line->{'unit_amount'}) ) {
# then show the unit amount and quantity
$OUT .=
'{\\dollar' . $line->{'unit_amount'} . '}'.
'{' . $line->{'quantity'} . '}';
} else {
# leave those columns blank
$OUT .= '{}{}';
}
$OUT .= '{\\dollar' . $line->{'amount'} . "}${rowbreak}\n";
}
$lastref = $line->{'ref'} || 0;

Output ext_description lines. We assume that any of them that contain an unescaped "&" are call details and should use \FScalldetail. Otherwise they use \FSextdesc.

foreach my $ext_desc (@$ext_description) {
if ($section->{extended_description_generator}) {
$OUT .= &{$section->{extended_description_generator}}($ext_desc);
} elsif ( $ext_desc !~ /[^\\]&/ ) {
$OUT .= '\FSextdesc{' . $ext_desc . "}$rowbreak\n";
} else { # call detail
$OUT .= '\FScalldetail{' . $ext_desc . "}$rowbreak\n";
}
}

}

End the longtable. If there is a posttotal item in this section (the final Balance Due / Credit Balance message, previous invoice aging, a few other things) then show it right-justified, in the same formatting used for the caption.

$OUT .= '\end{longtable}';
}
if ($section->{'posttotal'}) {
$OUT .= '\begin{flushright}';
$OUT .= '\normalfont\large\bfseries\textsc{'. $section->{'posttotal'}. '}\\\\';
$OUT .= '\\end{flushright}';
}
}

=== Notes ===

For the notes panel, create a minipage the full width of the text. (Unless $summary is in use; then the notes were on the summary page.)

There's some tricky handling of vertical space here. We want the notes at the bottom of the page, regardless of where the last section ended, so the \vfill between them acts as an expandable space.

However, if the notes end up on the first page, we have to prevent them from overprinting the coupon. So if there's a coupon and the notes are on page 1, we insert a strut of height \extracouponspace below the notes. The \vfill will then expand so that the bottom of the ''strut'' is at the bottom of the page, keeping the coupon space clear.

--@]
\vfill
\begin{minipage}[t]{\textwidth}
[@-- length($summary)
? ''
: ( $smallernotes
? '\scriptsize{ '.$notes.' }'
: $notes
)
--@]
[@-- $coupon ? '\ifthenelse{\equal{\thepage}{1}}{\rule{0pt}{\extracouponspace}}{}' : '' --@]
\end{minipage}
\end{document}

That's the end of the invoice.

== The summary template ==

Start with a two-column table, left side for the notes and right side for the billing summary. On the left side, make a 6.4cm minipage, and put a 10cm vertical strut there, along with the notes.

\begin{tabular}{ll}
\begin{minipage}{6.4cm}
\begin{tabular}{m{0cm}m{6.4cm}}
\rule{0cm}{10cm}&\begin{minipage}{6cm}[@-- $notes --@]\end{minipage}\\
\end{tabular}
\end{minipage} &

On the right side, place a 2cm horizontal strut followed by a 12.8cm minipage. This will contain the billing summary.

\rule{2cm}{0cm}
\begin{minipage}{12.8cm}

The billing summary itself is a two-column table, one for labels and one for amounts. The section headings are just labels with no amounts, with lines underneath.

'true_previous_balance' is, as far as we can determine it, the balance that was printed on the customer's previous invoice. It's the sum of all the customer's transactions (invoices, credits, payments, refunds) dated before that invoice, plus the invoice amount.

'balance_adjustments' is the sum of payments and credits received after the previous invoice, but applied to invoices before this current invoice. 'true_previous_balance' - 'balance_adjustments' is the total amount owed on all past invoices.

\begin{tabular}{lr}
\hline
&\\
\textbf{\underline{Summary of Previous Balance and Payments}} & \\
&\\
\textbf{Previous Balance}&\textbf{\dollar[@-- $true_previous_balance --@]}\\
\textbf{Payments}&\textbf{\dollar[@-- $balance_adjustments --@]}\\
\cline{2-2}
\textbf{Balance Outstanding}&\textbf{\dollar[@-- sprintf('%.2f', $true_previous_balance -$balance_adjustments) --@]}\\
&\\
\hline
&\\
\textbf{\underline{Summary of New Charges}} & \\
&\\

@summary_subtotals contains one line for each invoice section, except for the sections that have special handling such as the adjustments and finance charges. Make a line for each of those subtotals. This is followed with \cline{2-2}, which puts a horizontal line under the second column only, and then a row for the total of new charges.

[@--
foreach my $section (@summary_subtotals) {
$OUT .= '\textbf{'. ($section->{'description'} ? $section->{'description'} : 'Charges' ). '}';
$OUT .= '&\textbf{'. $section->{'subtotal'}. '}\\\\';
}
$OUT .= '\cline{2-2}';
--@]
\textbf{New Charges Total}&\textbf{\dollar[@-- $current_less_finance --@]}\\
&\\
\hline
&\\
\textbf{\underline{Invoice Summary}} & \\
& \\
\textbf{Previous Past Due Charges}&\textbf{\dollar[@-- sprintf('%.2f', $true_previous_balance - $balance_adjustments) --@]}\\
\textbf{Finance charges on overdue amount}&\textbf{\dollar[@-- $finance_amount --@]}\\
\textbf{New Charges}&\textbf{\dollar[@-- $current_less_finance --@]}\\

Then show the subtotal of the adjustment section, if there is one. This contains payments and credits applied to the ''current'' invoice.

[@--
#false laziness w/invoice_htmlsummary and above
foreach my $section ( grep $_->{adjust_section}, @sections ) {
$OUT .= '\textbf{'. ($section->{'description'} ? $section->{'description'} : 'Charges' ). '}';
$OUT .= '&\textbf{'. $section->{'subtotal'}. '}\\\\';
}
--@]

Finally, the customer's balance including the current invoice, and then close the table and minipage, and force a page break so the line items can start on a new page.

\cline{2-2}
\textbf{Total Amount Due}&\textbf{\dollar[@-- sprintf('%.2f', $balance) --@]}\\
&\\
\hline
\end{tabular}
\end{minipage} \\
\end{tabular}
\newpage

== Invoice modes ==

Freeside:4:Documentation:InstallingOnDebian8

2016-09-14T20:05:13Z

Mark: Change options for createuser for deb8

= Configure package repositories =
* Add the following apt sources to <code>/etc/apt/sources.list</code> (for Debian 8.x "jessie"):

<pre>
deb http://freeside.biz/~ivan/freeside-jessie/ ./
deb http://freeside.biz/~jeremyd/freeside4-jessie-stable/ ./
</pre>

* Run <code>aptitude update</code>.

= Install =

<pre>
aptitude install freeside freeside-lib freeside-webui
apt-mark hold freeside*
</pre>

= Database setup =

== Database User ==
* Allow the freeside user full access to the freeside database.
with Postgresql:
<pre>
[ as postgres/pgsql user ]
$ createuser -P -d freeside
Enter password for new role:
Enter it again:
</pre>
or with MySQL:
<pre>
$ mysqladmin -u root password 'set_a_root_database_password'
$ mysql -u root -p
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,INDEX,ALTER,CREATE,DROP on freeside.* TO freeside@localhost IDENTIFIED BY 'set_a_freeside_database_password';
</pre>

== Database Access ==
* Configure /usr/local/etc/freeside/secrets if necessary. This file contains three lines: DBI datasource, username and password
** See the DBI manpage and the manpage for your DBD for the exact syntax of your DBI data source.

== Database ==
* Add the freeside database to your database engine:
with Postgres:
<pre>
$ su freeside
$ createdb -E UTF8 freeside
</pre>
or with MySQL:
<pre>
$ mysqladmin -u freeside -p create freeside
</pre>

= Bootstrap Freeside =

== Freeside database ==

* As the freeside UNIX user, run <code>freeside-setup -d your.domain.name</code> to create the database tables and initial data.
<pre>
$ su freeside
$ freeside-setup -d example.com
</pre>

== RT database ==

* As the freeside UNIX user, run:
<pre>
$ su freeside
$ /opt/rt3/sbin/rt-setup-database --action schema
$ /opt/rt3/sbin/rt-setup-database --action coredata
$ /opt/rt3/sbin/rt-setup-database --action insert --datafile /opt/rt3/etc/initialdata
</pre>

== System users ==
* Create the Freeside system users:
<pre>
$ su freeside
$ freeside-adduser -g 1 fs_queue
$ freeside-adduser -g 1 fs_daily
$ freeside-adduser -g 1 fs_selfservice
$ freeside-adduser -g 1 fs_api
</pre>

== Employees ==
* Create one or more Freeside users (your internal sales/tech folks, not customer accounts):
<pre>
$ su freeside
$ freeside-adduser -g 1 username
$ htpasswd /usr/local/etc/freeside/htpasswd username
Password:
</pre>
Additional users can be added using the same command or from the web interface.

= Restart freeside =

* <code>/etc/init.d/freeside restart</code>

= Apache & Web GUI =
* Edit /etc/apache2/envvars and set APACHE_RUN_USER and APACHE_RUN_GROUP to <code>freeside</code>
* <code>a2enconf freeside-base2</code>
* <code>a2enconf freeside-rt</code>
* <code>a2dismod mpm_event</code>
* <code>a2enmod mpm_prefork</code>
* <code>chown freeside /var/lock/apache2</code>
* Restart Apache
* The web interface will be available at /freeside

= Next steps =
* Log into the web interface using the username and password you entered above.
* Proceed to the initial [[Freeside:3:Documentation:Administration|administration]] of your installation.

Freeside:Configuration:invoice-all pkg addresses

2016-07-13T00:12:15Z

Mark: Created page with "Category:Freeside_Configuration_Settings Category:Invoice_Layout ;invoice-all_pkg_addresses :Show all package addresses on invoices, even the default."

[[Category:Freeside_Configuration_Settings]]
[[Category:Invoice_Layout]]
;invoice-all_pkg_addresses
:Show all package addresses on invoices, even the default.

Freeside:3:Documentation:Administration:invoice layout

2016-07-13T00:10:54Z

Mark: /* Boolean Rendering Variables */ invoice-all_pkg_addresses

===Invoice Layout and Content===
====Basic Invoice Styles====
Several variables make significant changes to the appearance of invoices.

{{Freeside:Configuration:invoice_sections}}
{{Freeside:Configuration:invoice_usesummary}}
{{Freeside:Configuration:usage_class_as_a_section}}
{{Freeside:Configuration:svc_phone_sections}}

====Invoice Templates====
{{Freeside:3:Documentation:Administration:invoice_templates|format=latex}}

{{Freeside:3:Documentation:Administration:VOIP plan invoice layout variables}}

==== Other Variables Changing Invoice Layout at Billing Time====
{{Freeside:Configuration:separate_usage}}

==== Variables Neither Perfectly Rendering nor Perfectly Generational====
{{Freeside:Configuration:date_format}}
{{Freeside:Configuration:money_char}}
{{Freeside:Configuration:invoice_default_terms}}

====Boolean Rendering Variables====
{{Freeside:Configuration:invoice_show_prior_due_date}}
{{Freeside:Configuration:invoice_include_aging}}
{{Freeside:Configuration:invoice_smallernotes}}
{{Freeside:Configuration:invoice_smallerfooter}}
{{Freeside:Configuration:invoice-ship_address}}
{{Freeside:Configuration:invoice-all_pkg_addresses}}
{{Freeside:Configuration:disable_line_item_date_ranges}}
{{Freeside:Configuration:disable_previous_balance}}
{{Freeside:Configuration:previous_balance-summary_only}}
{{Freeside:Configuration:balance_due_below_line}}
{{Freeside:Configuration:invoice-unitprice}}
{{Freeside:Configuration:previous_balance-section}}

====Other Rendering Variables====
{{Freeside:Configuration:invoice_latextopmargin}}
{{Freeside:Configuration:invoice_latexheadsep}}
{{Freeside:Configuration:invoice_latexaddresssep}}
{{Freeside:Configuration:invoice_latextextheight}}
{{Freeside:Configuration:invoice_latexextracouponspace}}
{{Freeside:Configuration:invoice_latexcouponfootsep}}
{{Freeside:Configuration:invoice_latexcouponamountenclosedsep}}
{{Freeside:Configuration:invoice_latexcoupontoaddresssep}}
{{Freeside:Configuration:invoice_latexverticalreturnaddress}}
{{Freeside:Configuration:invoice_latexcouponaddcompanytoaddress}}
{{Freeside:Configuration:previous_balance-exclude_from_total}}
{{Freeside:Configuration:company_name}}
{{Freeside:Configuration:company_address}}
{{Freeside:Configuration:finance_pkgclass}}
{{Freeside:Configuration:cust_bill-max_same_services}}
{{Freeside:Configuration:cust_bill-consolidate_services}}

====Miscellaneous Variables Impacting Customer Perception of Invoices====

These variables do not change the invoice per se, but do change what is emailed to a customer as an invoice.
{{Freeside:Configuration:invoice_email_pdf}}
{{Freeside:Configuration:invoice_email_pdf_note}}
{{Freeside:Configuration:voip-cust_email_csv_cdr}}

Freeside:4:Documentation

2016-06-28T20:42:56Z

Mark: Reverted edits by Mark (talk) to last revision by Ivan

= Note =

Documentation specific to git master / 4.x. For now, refer to this as an add-on to the 3.x documentation.

= Installation and upgrades =

The minimum Perl version is now 5.10.

== Backend Installation ==

* New installation
* Integrated RT installation
* OS-specific installation guides (source)
** [[Freeside:4:Documentation:InstallingOnDebian8]]

== Upgrading ==

* [[Freeside:4:Documentation:Upgrading|Upgrading from 3.3 or later to 4.x]]

= Administrator =

* [[Freeside:4:Documentation:Administrator:Multi-currency|Multi-currency]]
* [[Freeside:4:Documentation:Administrator:Fees|Automated fees]]
* [[Freeside:4:Documentation:Cacti|Cacti Integration]]
* [[Freeside:4:Documentation:MagicMail|MagicMail Integration]]

= Developer =

* [[Freeside:4:Documentation:Developer:Authentication_Plugins|Authentication Plugins]]
* [[Freeside:4:Documentation:TaxEngine|Tax Engines]]

= Changelog =

* [[Freeside:4.0:Changelog|4.0 Changelog]]

= New feature documentation =

* [[Freeside:4:Documentation:Appointments]]
* Emails triggered by system log events can be set up at Configuration -> Miscellaneous -> System log emails

Freeside:4:Documentation

2016-06-28T19:17:36Z

Mark:

= Note =

Documentation specific to git master / 4.x. For now, refer to this as an add-on to the 3.x documentation.

= Installation and upgrades =

The minimum Perl version is now 5.10.

== Backend Installation ==

* New installation
* Integrated RT installation
* OS-specific installation guides (source)
** [[Freeside:4:Documentation:InstallingOnDebian8]]

== Upgrading ==

* [[Freeside:4:Documentation:Upgrading|Upgrading from 3.3 or later to 4.x]]

= Administrator =

* [[Freeside:4:Documentation:Administrator:Multi-currency|Multi-currency]]
* [[Freeside:4:Documentation:Administrator:Fees|Automated fees]]
* [[Freeside:4:Documentation:Administrator:SureTax|SureTax]]
* [[Freeside:4:Documentation:Cacti|Cacti Integration]]
* [[Freeside:4:Documentation:MagicMail|MagicMail Integration]]

= Developer =

* [[Freeside:4:Documentation:Developer:Authentication_Plugins|Authentication Plugins]]
* [[Freeside:4:Documentation:TaxEngine|Tax Engines]]

= Changelog =

* [[Freeside:4.0:Changelog|4.0 Changelog]]

= New feature documentation =

* [[Freeside:4:Documentation:Appointments]]
* Emails triggered by system log events can be set up at Configuration -> Miscellaneous -> System log emails

Freeside:4:Documentation

2016-06-28T19:17:20Z

Mark:

= Note =

Documentation specific to git master / 4.x. For now, refer to this as an add-on to the 3.x documentation.

= Installation and upgrades =

The minimum Perl version is now 5.10.

== Backend Installation ==

* New installation
* Integrated RT installation
* OS-specific installation guides (source)
** [[Freeside:4:Documentation:InstallingOnDebian8]]

== Upgrading ==

* [[Freeside:4:Documentation:Upgrading|Upgrading from 3.3 or later to 4.x]]

= Administrator =

* [[Freeside:4:Documentation:Administrator:Multi-currency|Multi-currency]]
* [[Freeside:4:Documentation:Administrator:Fees|Automated fees]]
* [[Freeside:4:Documentation:Administrator:Fees|SureTax]]
* [[Freeside:4:Documentation:Cacti|Cacti Integration]]
* [[Freeside:4:Documentation:MagicMail|MagicMail Integration]]

= Developer =

* [[Freeside:4:Documentation:Developer:Authentication_Plugins|Authentication Plugins]]
* [[Freeside:4:Documentation:TaxEngine|Tax Engines]]

= Changelog =

* [[Freeside:4.0:Changelog|4.0 Changelog]]

= New feature documentation =

* [[Freeside:4:Documentation:Appointments]]
* Emails triggered by system log events can be set up at Configuration -> Miscellaneous -> System log emails

Freeside:3:Documentation:User

2016-05-24T20:29:57Z

Mark: /* Tools */

= Introduction =

== About ==

http://www.freeside.biz/~ivan/freeside-slides/html/slide_1.html ?

AGPL now

== Navigation and Preferences ==

http://www.freeside.biz/~ivan/freeside-slides/html/slide_2.html

== Customers, Packages and Services ==

http://www.freeside.biz/~ivan/freeside-slides/html/slide_3.html

http://www.freeside.biz/~ivan/freeside-slides/html/slide_4.html (simple case: single package and service)

= Prospects =

== Adding prospcts ==

== Searching for prospects ==

== Prospect view ==

=== Qualifications ===

If you have any qualification-capable services configured, you may perform qualifications using the "New Qualification" link. Previously-performed qualifications may be viewed from "View Qualifications". See [[#Qualifications 2|Package Qualifications]] for details.

= Customers =

== Adding customers ==

http://www.freeside.biz/~ivan/freeside-slides/html/slide_5.html

== Searching for customers ==

== Customer view ==

http://www.freeside.biz/~ivan/freeside-slides/html/slide_6.html

=== Actions ===

==== Edit customer ====
==== Cancel customer ====
==== Refer customer ====
==== View customer's referrals ====
==== Bill now ====

=== Comments / Notes ===

==== Comments ====

==== Notes ====

=== Tickets ===

http://www.freeside.biz/~ivan/freeside-slides/html/slide_7.html

=== Packages ===

http://www.freeside.biz/~ivan/freeside-slides/html/slide_11.html

==== Package Dates ====

There are several dates which can be set on a package for a wide range of functionality

* Start date - Delay the package's first billing until a date in the future. Setting this in the past will have no effect and the package will bill immediately.
* Setup date - The first day the package bills. This also controls the billing of setup fees. Once a setup date is established on a package the setup fee will not bill

* Last bill date - The last date the package billed

* Next bill date - The next date the package will bill.

* Contract end date - Notes the end date of the customers current contract. Does not have any billing implications unless you setup billing events to suspend packages which have a lapsed contract end date. This date can also be used in the advanced package reports for auditing purposes.

It is not advised to manually edit last or next bill dates as editing these incorrectly can have undesired results.

'''NOTE: Package date editing will deprecated in 4.x in favor of specific work flow functionality.'''

==== Qualifications ====

If you have any qualification-capable services configured, you may perform qualifications using the "New Qualification" link under Packages, and then choosing the "Qualify using" field appropriately and filling out the form.

Previously-performed qualifications may be viewed from "View Qualifications".

Notes specific to New Qualification for Ikano:
# An address is always required
# Location Type may be only one of the fixed Ikano values - see their documentation
# Location Kind must always be chosen
# Dry loops - always leave "Service Telephone Number" empty
# Line-share (non dry loops) - always fill in "Service Telephone Number"

Notes specific to qualification results / viewing previous qualifications for Ikano:
# The view qualification page will show all possible packages in Freeside which you can order, based on the customer data which was qualified
# Click on the package to order

==== Services ====

===== Accounts =====

http://www.freeside.biz/~ivan/freeside-slides/html/slide_12.html

===== Domains =====
===== Mail Forwards =====
===== Virtual hosting =====
===== Broadband =====
===== Phone numbers =====
===== External =====
===== DSL =====

Ikano-specific notes:
* New Orders
# Perform a qualification (see Qualifications above)
# Order one of the qualifying packages shown on the qualification result and set the start date and location on this package appropriately
# For dry loops, leave the Service Telephone Number blank and choose the "Standalone" Loop Type
# For line-share (non dry-loops), always fill in the Service Telephone Number and choose the "Line-share" Loop Type
* Viewing DSL Orders - click on the service to view all order data, including notes placed on the order by Ikano or you via their web portal
* Canceling DSL Orders
** A NEW order in NEW or PENDING statuses may be canceled immediately - do a normal "Cancel now" on the package
** A NEW order in COMPLETED status may be canceled by expiring the package - do a normal "Cancel later" on the package with a cancel date at least 48 hours in future
* Changing PPPoE password - edit the service and change the password
* Suspending and unsuspending - do a normal Suspend Now or Unsuspend on the package
* Processes not supported currently (use the Ikano web interface for now):
** Canceling NEW orders in any status other than NEW, PENDING, or COMPLETED
** Anything involving CHANGE orders - e.g. changing the package/service, canceling a change/move, performing a move
** Aborting cancellations (unexpiring packages) while a CANCEL order is in NEW or PENDING status
** Syncing an order's disconnect or activation date to the package dates
* Due to Ikano's API, it will not be possible to place notes on an order from Freeside, so this cannot be implemented

=== Payment history and actions ===

http://www.freeside.biz/~ivan/freeside-slides/html/slide_13.html

==== Invoices ====

Invoices are generated by the system. The charges on each invoice reflect the setup/one-time, recurring and usage charges of that customer's packages. Invoices increase the customer's balance.

To add new packages or charges to a customer, use the "Order new package" or "One-time charge" links in the Packages section of the Customer View page.

To generate a pending invoice for a customer, use the "Bill now" link at the top of the Customer View page.

==== Payments ====

Payments are money the customer pays. Payments decrease the customer's balance.

===== Check and Cash payments =====

* To post a check or cash payment, use the "Enter check payment" or "Enter cash payment" links at the top of the Payment History section of the customer view page.
** Enter the amount of the payment.
** For check payments, enter the check number.
** Leave "Auto-apply to invoices" set to "yes" if you would like the system to apply the payment to any open invoices (oldest first). Or change "Auto-apply to invoices" to "no" if you would like to apply the payment to invoice(s) (or refund(s)) manually.

* Multiple check payments can also be entered in a batch under Tools -> Quick Payment Entry.

===== Credit card and electronic check (ACH) payments =====

* Credit card or electronic check payments will be initiated by the system for customers with Billing Type "Credit card (automatic)" or "Electronic Check (automatic)".

* Payments can be initiated manually by using the "Process credit card payment" or "Process electronic check (ACH) payment" links, in the Payment History section of the customer view page.
** Enter the amount to process (defaults to the customer's current outstanding balance).
** For credit card payments, enter the card number, expiration date, name on card and billing address. If the customer has a card on-file, the masked card number ("4111xxxxxxxx1111") and other information can be left as-is to charge the on-file card.
** For electronic check payments, enter the account number, account type, ABA or routing number, and bank name (other information such as bank state, social security number, or driver's license number may also be required by some payment gateways). If the customer has electronic checking information on-file, the masked account number and other information can be left as-is to charge the on-file bank account.
** Check "Remember this information" to save the payment information on-file.
** Also check "Charge future payments to this credit card/electronic check automatically" to charge the customer automatically in the future.

* Customers may use the self-service interface to pay by credit card or electronic check themselves.

===== Payment application =====

* Payments are applied to a specific invoice (or split between multiple invoices).
* The "(apply)" and "(unapply)" links next to specific payment (in the Payment History section of the customer view page) can be used to change the application of payments to specific invoice(s) and/or refunds(s).

==== Credits ====

Credits are adjustments to the amount the customer owes. Credits decrease the customer's balance.

* To post a credit, use the "Enter credit" link at the top of the Payment History section of the customer view page.
** Enter the amount to credit.
** Select the reason for the credit or enter a new reason.
** Leave "Auto-apply to invoices" set to "yes" if you would like the system to apply the credit to any open invoices (oldest first). Or change "Auto-apply to invoices" to "no" if you would like to apply the credit to invoice(s) (or refund(s)) manually.

* Like payments, credits can be applied to an invoice (or split between multiple invoices). For example, a credit to reverse an incorrect invoice or write off an invoice as bad debt could be applied to that specific invoice.
* Credits can also be applied to refunds to indicate that the credit was refunded to the customer instead.
* The "(apply)" and "(unapply)" links next to specific credits (in the Payment History section of the customer view page) can be used to change the application of credits to specific invoice(s) and/or refunds(s).

==== Refunds ====

Refunds are money paid to the customer. Refunds increase the customer's balance.

===== Check and Cash refunds =====

* Posting refunds
* Applying and unapplying refunds.

===== Credit card and electronic check (ACH) refunds =====

* To refund a specific credit card or electronic check (ACH) payment back to the customer, use the "(refund)" link next to that specific payment (in the Payment History section of the customer view page). This will process a refund for the amount of the payment with the payment gateway.

= Ticketing =

Link to RT documentation

== New ticket ==

http://www.freeside.biz/~ivan/freeside-slides/html/slide_8.html

== Ticketing Main ==

http://www.freeside.biz/~ivan/freeside-slides/html/slide_9.html

== Ticket View ==

http://www.freeside.biz/~ivan/freeside-slides/html/slide_10.html

= Searching and Reporting =

http://www.freeside.biz/~ivan/freeside-slides/html/slide_14.html

== Customers ==

* To search for a customer, enter the customer number, name, company name or contact phone number into the "Search customers" box at the top of each page. Name and company are "fuzzy" searches that will attempt to find a close match if no exact match is found.
* Other customer browsing and reporting is available from Reports -> Customers
* For advanced reporting with specific criteria, use the "Advanced" link next to the "Search customers" box at the top of each page, or go to Reports -> Customers -> Advanced customer reports

* To search for a customer by invoice number or service information (such as username, email address, domain or service phone number), use the invoice or service search, then click on the customer or "View this customer" link.

== Invoices ==

* To search for an invoice, enter the invoice number into the "Search customers" box at the top of each page.
* Other invoice browsing and reporting is available from Reports -> Invoices
* For advanced reporting with specific criteria, use the "Advanced" link next to the "Search invoice" box at the top of each page, or go to Reports -> Invoices -> Advanced invoice reports

== Packages ==

* Package browsing and reporting is available from Reports -> Packages
* For advanced reporting with specific criteria, go to Reports -> Packages -> Advanced package reports

=== FCC Form 477 ===

* See [[Freeside:3:Documentation:User:Form_477]].

== Services ==

* To search for a service such as username, email address, IP address, MAC address, domain or service phone number, enter the search data into the "Search servcies" box at the top of each page.
* Other service browsing and reporting is available from Reports -> Services

=== Accounts ===
=== Domains ===
=== Mail Forwards ===
=== Virtual hosting ===
=== Broadband ===
=== Phone numbers ===
=== External ===

== Usage ==

== Tickets ==

* To search for a ticket, enter the ticket number, subject, or email address into the "Search tickets" box at the top of each page.
* A fulltext ticket search can also be performed by entering "fulltext:searchstring" where searchstring is the string to search for.
* For advanced reporting with specific criteria, use the "Advanced" link next to the "Search tickets" box at the top of each page.

== Financial Reports ==

=== Sales, Credits and Receipts ===

=== Sales Report ===

=== Credit Report ===

=== Payment Report ===

=== Payment Batch Report ===

=== A/R Aging ===

Go to Reports -> Financial -> A/R aging to pull up an aging report

* Select "All customers" to pull up a report including all customers.
* Select "Customers with a balance" (the default) to pull up a report only including customers with a balance. Optionally enter a number of days to pull up a report only including customers with balances over the specified number of days old.

=== Prepaid income ===

=== Sales tax liability ===

* See [[Freeside:3:Documentation:User:Tax_liability_report]].

= Tools =

== Quick Payment Entry ==

== Tower coverage estimation ==

[[Freeside:3:Documentation:User:Tower_coverage_estimation|Enter your tower locations and display their expected coverage area.]]

= Auditing Legacy Data =

http://www.freeside.biz/~ivan/freeside-slides/html/slide_15.html

Freeside:3:Documentation:User:Tower coverage estimation

2016-05-24T20:22:07Z

Mark: Created page with "== Tower coverage estimation == Freeside now has simple capabilities to estimate the coverage areas of RF towers. To use this, you will need: * A tower and sector entry for t..."

== Tower coverage estimation ==

Freeside now has simple capabilities to estimate the coverage areas of RF towers. To use this, you will need:
* A tower and sector entry for the radio site, with coordinates and mast height.
* The antenna direction (0 - 360 degrees) and downtilt.
* The horizontal and vertical beamwidth of the antenna.
* A minimum signal margin for a site to be considered "covered". This is generally (transmit power) + (antenna gains) - (minimum RSSI). Most WISP operators should be able to determine this.

=== Generating a map ===
* Open the tower list (Configuration / Services / Wireless broadband / Towers).
* Click "Sector coverage maps".
* Find the entry for your tower.
** If any of the required fields are missing, the column to the right will list them. If so, click the tower name to edit the tower and sector, then come back.
* Click "Create map". This may take some time as the terrain maps are downloaded.
* There will now be a link for "View map--X margin". Click that to open the map.

=== Limitations ===
* This only works in North America. SRTM data is available for anywhere on Earth, but right now the software doesn't know the other download paths.
* The map won't display unless there's at least one broadband service attached to the sector (because it's actually the svc_broadband map for the sector).
* We can only show one signal margin on the map at a time.
* Similarly, we only show the map for one sector at a time. It would be nice to at least show the entire tower.
* All of these can be fixed.

Freeside:3:Documentation:Administration:RT Workflow

2015-12-15T20:59:22Z

Mark:

==Subtasks==

[[File:setup_subtasks.png|500px|right]]
Freeside 3.8 and above have a streamlined interface for setting up "Create Tickets" scrips in the typical case where some type of ticket will have a standard set of dependent tasks. To define subtasks for a ticket queue:

# Create a queue for this type of ticket, if you don't have one yet. Each queue can have only one set of subtasks.
# In the queue configuration, open the "Subtasks" tab on the top bar.
# For the first subtask, enter the subject and message for the dependent ticket.
#* Optionally, you can set the tickets to be created in a different queue.
#* You can check the "Prefix with main subject" box to have the subject line prefixed with the subject from the parent ticket.
# Click "Save changes".
# The page will refresh with space for a second subtask. Repeat as necessary for each subtask you want to create.

When adding a ticket to the queue, there will be a dropdown box labeled "Create subtasks". Set this to "Yes" and subtask tickets will be created. You can also create the ticket first, then set this field to "Yes" later to start the subtasks.
[[File:create_ticket.png|500px|right]]

=== Implementation ===

Internally, setting up subtasks creates the following objects:
* A ticket custom field named "Create subtasks", applied to the queue. This controls activation of subtasks on a per-ticket basis.
* A scrip condition that tests that custom field and activates when its value is set to "Yes".
* A template for the "CreateTickets" scrip action, containing the subtask subject and content and some standard ticket parameters.
** In particular, the subtask always inherits the parent ticket's requestor and owner.
** The parent ticket always has the subtask ticket added as a dependency, so it can't be resolved until all subtasks are complete.
* A scrip using that scrip condition and template. The scrip will trigger when the ticket is created with "Create subtasks = Yes", or when "Create subtasks" is set to "Yes" by a later transaction.

The subject and content can include Perl code in { braces }. The main ticket is available as the variable "$TOP" (an RT::Ticket object).

File:Create ticket.png

2015-12-15T20:54:09Z

Mark:

File:Setup subtasks.png

2015-12-15T20:53:31Z

Mark:

Freeside:3:Documentation:Administration:RT Workflow

2015-12-15T20:53:04Z

Mark: Documentation for RT subtasks

==Subtasks==

Freeside 3.8 and above have a streamlined interface for setting up "Create Tickets" scrips in the typical case where some type of ticket will have a standard set of dependent tasks. To define subtasks for a ticket queue:

# Create a queue for this type of ticket, if you don't have one yet. Each queue can have only one set of subtasks.
# In the queue configuration, open the "Subtasks" tab on the top bar.
# For the first subtask, enter the subject and message for the dependent ticket.
#* Optionally, you can set the tickets to be created in a different queue.
#* You can check the "Prefix with main subject" box to have the subject line prefixed with the subject from the parent ticket.
# Click "Save changes".
# The page will refresh with space for a second subtask. Repeat as necessary for each subtask you want to create.

[[File:setup_subtasks.png]]

When adding a ticket to the queue, there will be a dropdown box labeled "Create subtasks". Set this to "Yes" and subtask tickets will be created. You can also create the ticket first, then set this field to "Yes" later to start the subtasks.
[[File:create_ticket.png]]

=== Implementation ===

Internally, setting up subtasks creates the following objects:
* A ticket custom field named "Create subtasks", applied to the queue. This controls activation of subtasks on a per-ticket basis.
* A scrip condition that tests that custom field and activates when its value is set to "Yes".
* A template for the "CreateTickets" scrip action, containing the subtask subject and content and some standard ticket parameters.
** In particular, the subtask always inherits the parent ticket's requestor and owner.
** The parent ticket always has the subtask ticket added as a dependency, so it can't be resolved until all subtasks are complete.
* A scrip using that scrip condition and template. The scrip will trigger when the ticket is created with "Create subtasks = Yes", or when "Create subtasks" is set to "Yes" by a later transaction.

The subject and content can include Perl code in { braces }. The main ticket is available as the variable "$TOP" (an RT::Ticket object).

Test Suite

2015-08-31T21:55:20Z

Mark:

= Freeside Test Suite =

== Overview ==

Freeside now has a small automated test suite, consisting of:

* a database image in a known state, containing randomly generated customers and services
* a script of URLs to load from the web server
* reference copies of those pages
* some simple tools to load the database image, fetch the URLs, and compare them to the reference copies

Currently the tests cover only a small, commonly used subset of the UI, and a relatively simple billing setup.

== Running the tests ==

"freeside-test-run" is the main test script. Currently there's only one test
plan, "ui_tests". freeside-test-run will:

* download all the URLs listed in the test plan into a directory in /tmp
* compare them to the reference versions with "diff -ur"
* write the output to "freeside_test.YYYYMMDD.diff"

The raw output directory will not be deleted, so you can examine the results
yourself.

If you want to do anything with the database besides compare the test results
to reference, run "freeside-test-start" by hand first. This will create a
database with the test image and start Apache with a fake time of one day after
the last bill. If there's an existing Freeside database, it will be renamed to
"freeside_YYYYMMDD" (the current date).

To put the existing database back in place, run "freeside-test-stop", then
restart Apache and any Freeside services.

== Updating the reference pages ==

The simplest way to update the reference copies of the test pages is

bin/freeside-test-start
bin/freeside-test-fetch -d ./share/output

(from the FS-Test source directory). If you're installing from a git repo,
this will overwrite the working tree with the newly downloaded test pages.
You can then use "git diff" to examine your changes and "git commit" to
commit them when you're done.

== Adding new tests ==

For UI tests, add the URL, minus the "http://hostname/freeside" prefix,
to share/ui_tests. Then run freeside-test-fetch to get the reference output
of the test and copy it to share/output (and add that file to git).

When adding any new test, make sure to run it twice on the same database
and code version to detect random or time-sensitive variations. These will
throw false alerts when anyone else uses your test, so you'll need to suppress
them somehow.

== What's in the test database ==

Two hundred customers with random nonsensical U.S. addresses. One third have
automatic checking, the rest have credit cards. 10% of the customers have
separate billing and service addresses.

Four package definitions (plus the unique "system domain"): annual domain,
monthly prorated account, monthly prorated broadband service, monthly
phone. Each package has one service except the phone package, which has a
quantity limit of 4. The two monthly prorated packages have setup fees.

Each customer has three of the four packages, started over the period from
2015-08-01 to 2016-03-01. Each package has a service configured (with a random
username, domain name, phone number, or IP address). Customers were invoiced
on the first day of each month during that period.

There's a billing event to run automatic credit card billing on eligible
invoices. The configured gateway is Business::OnlinePayment::Dummy, which
returns success for everything, so customers with credit cards will be fully
paid.

One user with unlimited access. Username "test", password "test".

There are no cancellations or suspensions, no credits, no taxes, no refunds,
no discounts, no exports, and no RT tickets. All of this stuff lies in the
bright shining future.

If you modify the test database, be sure to update this section also. Also
update the company name to "Freeside Test X.Y.Z" where X.Y is the Freeside
version number and Z is the test database version number, and set the same
value in freeside-test-start so that old test databases aren't used by
mistake.

Test Suite

2015-08-31T21:54:27Z

Mark:

= Freeside Test Suite =

== Overview ==

Freeside now has a small automated test suite, consisting of:

- a database image in a known state, containing randomly generated customers and services
- a script of URLs to load from the web server
- reference copies of those pages
- some simple tools to load the database image, fetch the URLs, and compare them to the reference copies

Currently the tests cover only a small, commonly used subset of the UI, and a relatively simple billing setup.

== Running the tests ==

"freeside-test-run" is the main test script. Currently there's only one test
plan, "ui_tests". freeside-test-run will:

- download all the URLs listed in the test plan into a directory in /tmp
- compare them to the reference versions with "diff -ur"
- write the output to "freeside_test.YYYYMMDD.diff"

The raw output directory will not be deleted, so you can examine the results
yourself.

If you want to do anything with the database besides compare the test results
to reference, run "freeside-test-start" by hand first. This will create a
database with the test image and start Apache with a fake time of one day after
the last bill. If there's an existing Freeside database, it will be renamed to
"freeside_YYYYMMDD" (the current date).

To put the existing database back in place, run "freeside-test-stop", then
restart Apache and any Freeside services.

== Updating the reference pages ==

The simplest way to update the reference copies of the test pages is

bin/freeside-test-start
bin/freeside-test-fetch -d ./share/output

(from the FS-Test source directory). If you're installing from a git repo,
this will overwrite the working tree with the newly downloaded test pages.
You can then use "git diff" to examine your changes and "git commit" to
commit them when you're done.

== Adding new tests ==

For UI tests, add the URL, minus the "http://hostname/freeside" prefix,
to share/ui_tests. Then run freeside-test-fetch to get the reference output
of the test and copy it to share/output (and add that file to git).

When adding any new test, make sure to run it twice on the same database
and code version to detect random or time-sensitive variations. These will
throw false alerts when anyone else uses your test, so you'll need to suppress
them somehow.

== What's in the test database ==

Two hundred customers with random nonsensical U.S. addresses. One third have
automatic checking, the rest have credit cards. 10% of the customers have
separate billing and service addresses.

Four package definitions (plus the unique "system domain"): annual domain,
monthly prorated account, monthly prorated broadband service, monthly
phone. Each package has one service except the phone package, which has a
quantity limit of 4. The two monthly prorated packages have setup fees.

Each customer has three of the four packages, started over the period from
2015-08-01 to 2016-03-01. Each package has a service configured (with a random
username, domain name, phone number, or IP address). Customers were invoiced
on the first day of each month during that period.

There's a billing event to run automatic credit card billing on eligible
invoices. The configured gateway is Business::OnlinePayment::Dummy, which
returns success for everything, so customers with credit cards will be fully
paid.

One user with unlimited access. Username "test", password "test".

There are no cancellations or suspensions, no credits, no taxes, no refunds,
no discounts, no exports, and no RT tickets. All of this stuff lies in the
bright shining future.

If you modify the test database, be sure to update this section also. Also
update the company name to "Freeside Test X.Y.Z" where X.Y is the Freeside
version number and Z is the test database version number, and set the same
value in freeside-test-start so that old test databases aren't used by
mistake.

Test Suite

2015-08-31T21:54:07Z

Mark: Copy FS-Test/README, clean up a little

Main Page

2015-08-31T21:44:53Z

Mark:

== Freeside ==
=== Versions ===

:'''Stable Version:''' 3.7
::released Jun 26, 2015

:'''Development Version:''': 4.x
::2015

:'''Future''': 5.x / git master
::2016?

=== Turn Key Solutions ===
*[http://www.freeside.biz/freeside/services.html#install Installation]
*[http://www.freeside.biz/freeside/products.html Freeside Appliance]

=== Documentation ===

* [[Freeside:2.3:Documentation|2.3 Documentation]]
* [[Freeside:3:Documentation|3.x Documentation]]
* [[Freeside:4:Documentation|4.x Documentation]]

=== Support ===
*[http://www.freeside.biz/freeside/services.html#support Freeside Internet Services Inc.]
*[[Freeside:Support:Consultants | Freeside Consultants]]
*[[Freeside:Support:HelpWanted | Help Wanted]]

===Training===
*[[Training syllabus]]

=== Specs in progress ===
* [[Use Cases]]
* [[Business::FraudDetect]]
* [[Event_Refactor]]
* [[UI_Refactor]]
* [[Website_Refactor]]
* [[Browser_support]]
* [[Test Suite]]

=== Historical ===
* [[Batch_Refactor]]
* [[Broadband_Services_Spec]]
* [[Virtual_to_Real_Fields]]
* [[part_pkg Mixin Refactor]]
* A new [[Version_Control_System]]
* [[RPM_Build_system]]

= Third party software =

[[3rd_party_software|Third party software]] related to Freeside.

= WIKI Reference =

[http://meta.wikimedia.org/wiki/Help:Editing How to edit pages (wiki markup, etc)] 
[http://www.mediawiki.org/wiki/Help:Configuration_settings Configuration settings list] 
[http://meta.wikipedia.org/wiki/MediaWiki_User%27s_Guide User's Guide]

[[Sandbox]] ← Use this page to test out editing, [http://www.phrases.org.uk/meanings/225200.html learn the ropes], etc.

Main Page

2015-08-31T21:38:01Z

Mark: /* Specs in progress */

== Freeside ==
=== Versions ===

:'''Stable Version:''' 3.7
::released Jun 26, 2015

:'''Development Version:''': 4.x
::2015

:'''Future''': 5.x / git master
::2016?

=== Turn Key Solutions ===
*[http://www.freeside.biz/freeside/services.html#install Installation]
*[http://www.freeside.biz/freeside/products.html Freeside Appliance]

=== Documentation ===

* [[Freeside:2.3:Documentation|2.3 Documentation]]
* [[Freeside:3:Documentation|3.x Documentation]]
* [[Freeside:4:Documentation|4.x Documentation]]

=== Support ===
*[http://www.freeside.biz/freeside/services.html#support Freeside Internet Services Inc.]
*[[FreeSide:Support:Consultants | Freeside Consultants]]
*[[FreeSide:Support:HelpWanted | Help Wanted]]

===Training===
*[[Training syllabus]]

=== Specs in progress ===
* [[Use Cases]]
* [[Business::FraudDetect]]
* [[Event_Refactor]]
* [[UI_Refactor]]
* [[Website_Refactor]]
* [[Browser_support]]
* [[Testing Tool]]

=== Historical ===
* [[Batch_Refactor]]
* [[Broadband_Services_Spec]]
* [[Virtual_to_Real_Fields]]
* [[part_pkg Mixin Refactor]]
* A new [[Version_Control_System]]
* [[RPM_Build_system]]

= Third party software =

[[3rd_party_software|Third party software]] related to Freeside.

= WIKI Reference =

[http://meta.wikimedia.org/wiki/Help:Editing How to edit pages (wiki markup, etc)] 
[http://www.mediawiki.org/wiki/Help:Configuration_settings Configuration settings list] 
[http://meta.wikipedia.org/wiki/MediaWiki_User%27s_Guide User's Guide]

[[Sandbox]] ← Use this page to test out editing, [http://www.phrases.org.uk/meanings/225200.html learn the ropes], etc.

Freeside:4:Documentation:ExternalMessaging

2015-08-31T01:07:05Z

Mark: Some basic docs for this feature

== Creating the messaging service ==

The Freeside external messaging interface is a REST client. It communicates with the service by POSTing to specified URLs (which can be different for each kind of message, but need not be). The body of the POST is always a JSON object.

Messages are processed in two stages. At either stage the server can report failure by returning an HTTP failure status (non-200). The body of the HTTP response will be saved as the error message.

=== Prepare ===
Freeside POSTs a JSON object containing all of the same substitution variables available to internal message templates. See http://localhost/freeside/edit/msg_template/email.html for a list of these.
The element names in the object are the same as the variable names, minus the '$' sigil. The messaging service must return a document containing the "prepared message": everything it needs to successfully send the message later. This should be a JSON document, though that's not yet enforced (as of September 2015).

At this stage the user might still decide not to send the message, so the messaging service shouldn't do anything irreversible yet.

=== Send ===
Freeside POSTs the prepared message verbatim. At this point the messaging service should deliver it to the customer.

On success, the body of the response message will be ignored.

== Example ==

There's an example REST server in bin/msg_template_http-demo.pl, using Mojolicious. It shows how to take substitution variables, insert them into a body of text, and return a JSON object representing the message to be sent, and then (at the Send stage) how to take that object and turn it into an email message.

== Configuring the message interface in Freeside ==

From the menu bar, open '''Configuration -> Miscellaneous -> Message templates'''. Click the link for '''External message interfaces''', then '''Add a new interface'''.

'''Agent''' is which agent will have access to this interface (or "all" for it to be globally available).
'''Interface name''' is a descriptive name for your use.
'''Prepare URL''' and '''Send URL''' are the URLs to be POSTed to in the Prepare and Send stages, respectively. No variable substitution is performed on these.
'''HTTP username''' and '''password''' are a login for Basic authentication. This is optional.
'''Additional POST content''' is an optional JSON object (a complete object, including surrounding { } delimiters) to be included in the Prepare document. All the standard substitution variables will be added to it. If you don't specify one, you'll send an object containing only the standard variables.