network_inventory/docs/notes.org

#+TITLE: Network Inventory
:preamble:
#+author: Andreas Zweili
:end:

* Vocabulary

- Technician :: An employee working with the inventory tool and full access to
  its information.
- Customer :: A customer which owns some of the devices in the inventory tool
  and might have access to only the information related to the devices he owns.

* TODO Must Have [0/8]
** TODO Views [0/2]
*** NEXT add a button to add a relation            :computer_detail_view:

For the many to many relationships it might be a better idea to add them in a
separate form. So that the view displays the information but when you want to
add a new CPU for example it shows a button "Add CPU" which you can click an it
brings you to a new page where the PC is already preselected and the CPU can
get selected from a drop down.

*** NEXT implement SoftwareDetailView

I don't remember what the initial idea here was. We could show here
which customers are using this software. But that is currently a really low
priority item.

** TODO Forms [0/12]
*** NEXT Computer
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT Device
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT ConnectedDevice
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT Customer
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT Backup
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT UserLicense
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT DeviceLicense
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT Net
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT Software
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT Users
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT Groups
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT Fix the warranty in the device view

With currently the containers don't disappear fully. Should be easy to fix.

** TODO Various [0/12]
*** TODO change the admin url

For security reasons it's recommended to change the name of the admin panel
url. This way automated tools can't find it so easy. It only increases the
security slightly.

*** TODO update the url code

I'm currently not sure what I wanted to do with this.
https://docs.djangoproject.com/en/2.2/topics/http/urls/#views-extra-options

*** TODO implement guardian

This needs to be done for basically every model which lives on a view. E.g.
~BackupListView~, ~SoftwareListView~. I can’t remember how this should be
implemented. However it might be implemented in the customer table view. The
security concept works like this:
1. check if the user is logged in
2. check if the user is allowed to view the customer, if not return an error
3. Get all matching objects which the user is allowed to view. Step two can't
   be replaced by an empty table because we need a customer object to operate
   on. Therefore it's better to quickly check the customer before we fetch all
   the other objects from the database.

*** TODO calculate licence usage for customer

The view should show the licenses which the customer currently has available
and how many are already used. In addition it should show a visual warning to
the technician when the limit is reached.

*** TODO Convert the NETSheet Data file to YML fixtures.

A lot of this is already done. Only the hardware models are currently missing.

*** TODO Check tests for response.context[‚table‘]

This would allow for tests of the views which check explicitly what gets
returned by the view. Might be easier/faster then rendering the whole view.

However for some views it would be better to test the final view because the
template contains logic which can fail.

*** TODO Create custom user model

It is best practice to create a custom user model to allow future modifications
to the users without causing problems.

*** TODO [#C] Implement an excel import and export

might be achieved with this project:
- https://github.com/pyexcel-webwares/django-excel
This might be a nice to have feature but the copy button is more important to
achieve a similar funcition.

*** TODO Have a look at the documentation of django-nested-admin

I implemented nested-admin currently in a very basic way. I should read the
documentation in order to make sure that I'm using it correctly.

*** TODO have a look at django select_related, it might solve a problem for me.

I often find myself trying to get related objects. The method select_related
might help with that:
- https://docs.djangoproject.com/en/2.2/ref/models/querysets/#select-related

*** TODO Add the warranty to the Device, ConnectedDevice and Computer admin pages
*** TODO [#A] Add a feature to copy objects

This is a must have for the first release.

** TODO Licenses [0/2]
*** TODO Implement a license check into all forms

This should prevent technicians from assigning licenses which the customer has
already fully used.

*** TODO Add a check to see if a software has a license attached to it.

Add a check to see if a software has a license attached to it. I forgot the
reason why I need this. If so it increases the used licenses counter.
Maybe with this:
- https://stackoverflow.com/questions/23059088/manytomany-field-check-if-relation-exists

** TODO Computer [0/1]
*** NEXT cpu reduce required fields

The CPU has many required fields but sometimes they don't make sense. For
example when we have a virtual CPU we usually don't need to know the specific
frequenzy.

** TODO Backups [0/2]
*** NEXT Days need to be manytomany

A backup can be run on multiple days

*** NEXT a backup should be able to contain multiple computers

** TODO Groups [0/2]
*** NEXT Table for groups
*** NEXT the computer is missing a GPU

CAD computers often have sppecial graphics cards which we should be able to
track.

** TODO NETs [0/1]
*** NEXT the IP needs to be able to be null

Currently it's always required however when a device is in DHCP mode we can't
know the IP for sure.

* TODO Nice to Have [0/9]
*** TODO allow technicians to add custom fields

This would allow technicians to create custom models without change
Maybe this approach would be something:
- https://stackoverflow.com/questions/34907014/django-allow-user-to-add-fields-to-model

*** TODO Extend the CSS

- A more centered layout would be nice
- Maybe some colours

*** TODO calculate the used space on a host

Means calculate the size all the VMs would use if they were thick.
This could help a technician to properly plan ressources on a host.

*** TODO include a RAID calculator

I would like to use this to show the usable space in a RAID system. Currently
we enter this information by hand but it would be easier to calculate it
automatically.
- https://thoughtworksnc.com/2017/08/30/writing-a-raid-calculator-in-python

*** TODO Get warranty information from Dell

We sell a lot of Dell devices and it would be nice to use the service tags to
collect the warranty information directly from Dell. There's an API for that:
https://www.programmableweb.com/api/dell-warranty-status-rest-api

*** TODO Move the lists to their own page

Since I have more devices than I thought it would provide a better overview
than one big list. Forgot again what this exactly means.

*** TODO A "to deactivate" feature on inventory users

This way a technician could mark a user for deactivation and anyone could check
if there are users to deactivate. This would help if we would've to deactivate
a user at a certain date. The inventory tool could then show to all technicians
that the user needs to be deactivated. Then any technician could deactivate the
user and not just the technician responsible for the customer, increasing the
security of the customer.

*** TODO Add search boxes to the views.

So that one can search for a string in the responding column.

*** TODO Filter Hardware Model to corresponding device manufacturer

When changing the HardwareModel field of a device the dropdown should be
filtered to the provided DeviceManufacturer.

Currently it could still make sense to make the DeviceManufacturer only
available through the HardwareModel. This way we wouldn't have to filter the
HardwareModel dropdown. However we would loose the ability to only select the
DeviceManufacturer for a device in case we don't know the specific model which
happens quite often.


* Done
** DONE Recreate the RM in draw.io

The Dia RM is okay but not really that great. Draw.io would give a better
result.

** DONE create multiple requirements files
** DONE put passwords into environment variables
** DONE Permissions recherchieren
** DONE customer tabelle erweitern mit listen
** DONE Models erstellen
** DONE Add a Counter to the RAM Modules
** DONE Create a NET category where a device can live in.

This NET Category should display it's IP range, Subnet mask and show it's DHCP
range if one is configured.

** DONE Create class DeviceInNet

This class shows the relationship between the device and a NET. An attribute of
a DeviceInNet should be an IP address.

** DONE Create an abstract company class
** DONE Create Customer and a Manufacturer sub class Those two would be based on

the company class. I'm currently not sure how I should handle the case where a
company is both a customer and a manufacturer.

** DONE A text field next to the customer

where one can enter additional information which can't be put into the normal
documentation.

** DONE Fix test for net detail view
** DONE NETs, add a description field, for NETs like HEHImmo it might be nice to

have a short description for what it is intendet.

** DONE ComputerDetailView, add link to SoftwareDetailView
** DONE implement NETSheet list

this view should give an overview of all the devices in the NET and there
current IP Address.

** DONE implement BackupListView
** DONE Filter the queryset in the AllComputerView

so that it only shows the customers the current user is allowed to view

** DONE Disks in RAID and RAID have overlapping Felds (disks appear on both).

And they don't have the proper relationship. There can be disks from variing
sizes in a RAID therefore the relationship between DisksInRaid and
RaidInComputer needs to be a manytoone relationship

** DONE fix column name links in customer table

they throw an error when one clicks on them.

** DONE ComputerDetailView, add all properties to the view table
** DONE implement UserListView
** DONE implement SoftwareListView

this and the next view would probably better be a License view. Since the
software should be available to all devices from all customers. It doesn’t make
much sense to add 100 of different Office softwares. Probably a Software model
could be attached to a License model.

** DONE implement UserDetailView
** DONE Implement the license so that it can get attached to a user

when the user gets created. This way they might get less easily forgotten.

** DONE fix the Makefile so that the fixtures don't get applies twice.

This is already done for the ~make local~ command but needs fixing in the
~make~ command. However there's a bit more difficult because it runs in Docker
and with PostgreSQL

** DONE refactor the project to have a core app.
CLOSED: [2020-01-14 Tue 21:25]

This way I can split the project into multiple apps such as Customer, Computer,
Backups etc. and import the shared models from core. This allows me to split
the views and tests over multiple apps making the whole thing a bit easier to
understand. See the Notability note for more information.
https://github.com/netbox-community/netbox/tree/develop/netbox might provide an
example When doing the refactor I should correct the imports. The current
system is very annoying when I add a new object/class.

** DONE Hardware Model
CLOSED: [2020-02-14 Fri 20:28]

I'm currently unsure if I should implement a hardware model. With this model I
could add the hardware model to a device. Currently this capability is missing.

** DONE add a list of assigned users and computers to the license view
CLOSED: [2020-02-15 Sat 18:53]
** DONE Server mit NGINX aufsetzen
CLOSED: [2020-02-15 Sat 18:56]

- https://docs.djangoproject.com/en/2.2/howto/deployment/wsgi/uwsgi/
- https://uwsgi-docs.readthedocs.io/en/latest/tutorials/Django_and_nginx.html
- https://linuxconfig.org/how-to-host-django-with-nginx-on-ubuntu-18-04-bionic-beaver-linux

** DONE CustomerListView [3/3]
CLOSED: [2020-02-16 Sun 18:45]

add all the objects

- [X] Backup
- [X] Software
- [X] Users

** DONE implement permission decorators currently all the items can get viewed.
CLOSED: [2020-02-16 Sun 18:52]

I either have to implement a decorator for each object type or find a general
way. This problem is only related to detail views. The tables and lists have a
general way to check the permission. I maybe could get the model name from the
url, this Stackoverflow post might help:
- https://stackoverflow.com/questions/58307055/access-django-model-name-from-admin-url-pattern
and get the object with this function:
- https://stackoverflow.com/questions/28533174/programatically-accessing-django-models-from-another-app

** DONE Add tests for multiple nets and devices
CLOSED: [2020-02-16 Sun 18:52]
** DONE rename variables for the querysets to XXXRelations
CLOSED: [2020-02-16 Sun 18:53]

** DONE limit the queryset in the customer_table
CLOSED: [2020-02-16 Sun 19:13]

The queryset should only contain results which a users is allowed to see.

** DONE make sure the licenses models are correct.
CLOSED: [2020-02-16 Sun 19:31]

I think manytomany might not be the correct relation since a user should only
be attached once to a user license and a computer should only be attached once
to a computer license. However a user can stil have many licenses and a license
can still have many users.

** DONE tables problem
CLOSED: [2020-04-20 Mo 12:49]

This isn't fully working yet in django_tables2
https://github.com/jieter/django-tables2/issues/721

#+begin_src diff
-from django_tables2.utils import A
+

 class CustomersTable(tables.Table):
-    name = tables.LinkColumn('customer', args=[A('pk')])
-    nets = tables.LinkColumn('nets', text='Nets', args=[A('pk')])
-    computers = tables.LinkColumn('computers', text='Computers', args=[A('pk')])
-    devices = tables.LinkColumn('devices', text='Devices', args=[A('pk')])
-    backups = tables.LinkColumn('backups', text='Backups', args=[A('pk')])
+    name = tables.Column(linkify=("customer", [tables.A("pk")]))
+    nets = tables.Column(verbose_name="Nets",
+                         linkify=("nets", [tables.A("pk")]))
+    computers = tables.Column(verbose_name="Computers",
+                              linkify=("computers", [tables.A("pk")]))
+    devices = tables.Column(verbose_name="Devices",
+                            linkify=("devices", [tables.A("pk")]))
+    backups = tables.Column(verbose_name="Backups",
+                            linkify=dict(viewname="backups", args=[tables.A("pk")]))
#+end_src

** DONE implement a warranty overview
CLOSED: [2020-04-20 Mo 13:31]

2020-04-20
This is implementend now. I've setup the view so that customers can use the
view as well and the content gets limited to what they are allowed to see.

This view would show all devices which are running out of warranty, maybe this
could be shown as well in the CustomerDetailView. So that we would've a list
for the customers to see and one large list which shows the warranties for all
customers for internal usuage.

* Ressources
** Class Based Views

- http://ccbv.co.uk/

** Design
*** Admin themes
- django-grappelli
- django-suit
- django-admin-bootstrapped

** Forms

- https://django-crispy-forms.readthedocs.io/en/latest/index.html
- https://stackoverflow.com/questions/25321423/django-create-inline-forms-similar-to-django-admin*25340256
- https://stackoverflow.com/questions/5171365/django-inline-form-with-custom-forms

** Permissions

- https://django-guardian.readthedocs.io/en/stable/userguide/assign.html
- https://github.com/dfunckt/django-rules/blob/master/README.rst

#+begin_src python
decororator (function) :
  if user has permission(object.customer):
    return function
#+end_src

Maybe it would be possible to add a property to the classes which allows to
access the customer of an object like this:

#+begin_src python
object.customer
#+end_src

* Links to include

- https://docs.djangoproject.com/en/2.2/ref/models/querysets/#id4
- https://docs.djangoproject.com/en/2.2/ref/request-response/
- https://duckduckgo.com/?q=django+get_related&t=fpas&ia=qa
- https://pybit.es/selenium-pytest-and-django.html
- https://stackoverflow.com/questions/28533174/programatically-accessing-django-models-from-another-app
- https://stackoverflow.com/questions/54592026/how-to-create-a-custom-mixin-in-django
- https://stackoverflow.com/questions/58307055/access-django-model-name-from-admin-url-pattern
- https://stackoverflow.com/questions/6069070/how-to-use-permission-required-decorators-on-django-class-based-views#6069444